{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Anomaly Detection in Blockchain System with TODS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is TODS?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODS, developed by DATA Lab @ Texas A&M University, is a full-stack automated machine learning system for outlier detection on multivariate time-series data. TODS provides exhaustive modules for building machine learning-based outlier detection systems, including: data processing, time series processing, feature analysis (extraction), detection algorithms, and reinforcement module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is the Blockchain System?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\"A blockchain is, in the simplest of terms, a time-stamped series of immutable records of data that is managed by a cluster of computers not owned by any single entity. Each of these blocks of data (i.e. block) is secured and bound to each other using cryptographic principles (i.e. chain).\" \n",
    "\n",
    "More information about blockchain can be found at https://blockgeeks.com/guides/what-is-blockchain-technology/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How does TODS detect anomalies from Blockchain System?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODS system can perform three common outlier detection scenarios on time-series data: point-wise detection (time points as outliers), pattern-wise detection (subsequences as outliers), and system-wise detection (sets of time series as outliers).\n",
    "\n",
    "In this notebook, we use Google BigQuery Bitcoin Blockchain Dataset, which contains information about dates, transactions, blocks and prices of Bitcoin. According to Google, this dataset updates every 10 minutes in the following link:\n",
    "https://bigquery.cloud.google.com/dataset/bigquery-public-data:bitcoin_blockchain\n",
    "\n",
    "With TODS, we first collect and process the data to put in a dataframe. Next, we use two primitives from TODS to fit and predict on all dimensions of the data, to get the prediction labels and scores. Last, we use the prediction information from TODS primitives to plot and visualize the outliers within the data.\n",
    "\n",
    "The searcher of TODS will take the dataset and automatically build a pipeline to find the best possible model with its scores. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zV8tJwO6-U9h"
   },
   "source": [
    "## Packages and Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "mnF3OkUJ-U9i",
    "outputId": "af2dc0fd-9b87-4a0a-d95c-285f7101f654"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: google-cloud-bigquery in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (2.20.0)\n",
      "Requirement already satisfied: google-resumable-media<2.0dev,>=0.6.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.3.0)\n",
      "Requirement already satisfied: protobuf>=3.12.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (3.17.3)\n",
      "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (2.23.0)\n",
      "Requirement already satisfied: proto-plus>=1.10.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.18.1)\n",
      "Requirement already satisfied: google-cloud-core<2.0dev,>=1.4.1 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.6.0)\n",
      "Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.29.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (1.31.0)\n",
      "Requirement already satisfied: packaging>=14.3 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-bigquery) (21.0)\n",
      "Requirement already satisfied: six in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.15.0)\n",
      "Requirement already satisfied: google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\" in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.1.2)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (3.0.4)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (1.25.11)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (2021.5.30)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-cloud-bigquery) (2.10)\n",
      "Requirement already satisfied: google-auth<2.0dev,>=1.24.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (1.33.1)\n",
      "Requirement already satisfied: setuptools>=40.3.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (57.4.0)\n",
      "Requirement already satisfied: pytz in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (2021.1)\n",
      "Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (1.53.0)\n",
      "Requirement already satisfied: grpcio<2.0dev,>=1.29.0; extra == \"grpc\" in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-api-core[grpc]<2.0.0dev,>=1.29.0->google-cloud-bigquery) (1.36.1)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from packaging>=14.3->google-cloud-bigquery) (2.4.7)\n",
      "Requirement already satisfied: cffi>=1.0.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (1.14.0)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4; python_version >= \"3.6\" in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (4.7.2)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (0.2.8)\n",
      "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (4.2.2)\n",
      "Requirement already satisfied: pycparser in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from cffi>=1.0.0->google-crc32c<2.0dev,>=1.0; python_version >= \"3.5\"->google-resumable-media<2.0dev,>=0.6.0->google-cloud-bigquery) (2.20)\n",
      "Requirement already satisfied: pyasn1>=0.1.3 in /Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages (from rsa<5,>=3.1.4; python_version >= \"3.6\"->google-auth<2.0dev,>=1.24.0->google-cloud-core<2.0dev,>=1.4.1->google-cloud-bigquery) (0.4.8)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "pip install google-cloud-bigquery\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "hI2JIaHC-U9i",
    "outputId": "4424ee78-f84f-4e1c-9cdf-c0086ce3dca5",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d3m.primitives.tods.detection_algorithm.LSTMODetector: Primitive is not providing a description through its docstring.\n"
     ]
    }
   ],
   "source": [
    "from google.cloud import bigquery\n",
    "from scipy.stats.mstats import zscore\n",
    "from sklearn.metrics import precision_recall_curve\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import classification_report\n",
    "from tods.sk_interface.detection_algorithm.Telemanom_skinterface import TelemanomSKI\n",
    "from tods.sk_interface.detection_algorithm.DeepLog_skinterface import DeepLogSKI\n",
    "from sklearn.preprocessing import MinMaxScaler, QuantileTransformer\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn.manifold import TSNE\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "import matplotlib as mpl\n",
    "from pathlib import Path\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "import seaborn as sns\n",
    "import datetime as dt\n",
    "from datetime import datetime,tzinfo\n",
    "import scipy, json, csv, time, pytz\n",
    "from pytz import timezone\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "seed = 135\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "%matplotlib inline\n",
    "import os\n",
    "# os.listdir('./')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oFJ3d5MW-U9j"
   },
   "source": [
    "## Processing Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "1Laxlepi-U9j"
   },
   "outputs": [],
   "source": [
    "#Connecting to Google datastore (use path to ur private key)\n",
    "os.environ['GOOGLE_APPLICATION_CREDENTIALS']=\"blockchain-316820-a21123305db7.json\"\n",
    "client = bigquery.Client()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "ZkCH3LR5-U9k"
   },
   "outputs": [],
   "source": [
    "# The query to get date, number of transactions from Google BigQuery bitcoin blockchain dataset \n",
    "# Select records from the last three years and group them with respect to date\n",
    "query_1 = \"\"\"\n",
    "SELECT \n",
    "   DATE(TIMESTAMP_MILLIS(timestamp)) AS Date,\n",
    "   COUNT(transactions) AS Transactions\n",
    "FROM `bigquery-public-data.bitcoin_blockchain.blocks`\n",
    "GROUP BY date\n",
    "HAVING date >= '2009-01-09' AND date <= '2020-06-12'\n",
    "ORDER BY date\n",
    "\"\"\"\n",
    "query_job_1 = client.query(query_1)\n",
    "# Waits for the query to finish\n",
    "iterator_1 = query_job_1.result(timeout=30)\n",
    "rows_1 = list(iterator_1)\n",
    "df_1 = pd.DataFrame(data=[list(x.values()) for x in rows_1], columns=list(rows_1[0].keys()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "NXpxzJ5R-U9k"
   },
   "outputs": [],
   "source": [
    "# The query to get sum of all satoshis spent each day and number of blocks\n",
    "query_2 = \"\"\"\n",
    "SELECT\n",
    "  o.Date,\n",
    "  COUNT(o.block) AS Blocks,\n",
    "  SUM(o.output_price) AS Output_Satoshis\n",
    "FROM (\n",
    "  SELECT\n",
    "    DATE(TIMESTAMP_MILLIS(timestamp)) AS Date,\n",
    "    output.output_satoshis AS output_price,\n",
    "    block_id AS block\n",
    "  FROM\n",
    "    `bigquery-public-data.bitcoin_blockchain.transactions`,\n",
    "    UNNEST(outputs) AS output ) AS o\n",
    "GROUP BY\n",
    "  o.date\n",
    "HAVING o.date >= '2009-01-09' AND o.date <= '2020-06-12'\n",
    "ORDER BY o.date, blocks\n",
    "\"\"\"\n",
    "query_job_2 = client.query(query_2)\n",
    "# Waits for the query to finish\n",
    "iterator_2 = query_job_2.result(timeout=30)\n",
    "rows_2 = list(iterator_2)\n",
    "df_2 = pd.DataFrame(data=[list(x.values()) for x in rows_2], columns=list(rows_2[0].keys()))\n",
    "\n",
    "df_2[\"Output_Satoshis\"]= df_2[\"Output_Satoshis\"].apply(lambda x: float(x/100000000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "TQzZev9C-U9k",
    "outputId": "c643c8e9-b222-4ce9-f306-ae3a8812988d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Transactions</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2009-01-09</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2009-01-10</td>\n",
       "      <td>61</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2009-01-11</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2009-01-12</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2009-01-13</td>\n",
       "      <td>123</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date  Transactions\n",
       "0  2009-01-09            14\n",
       "1  2009-01-10            61\n",
       "2  2009-01-11            93\n",
       "3  2009-01-12            94\n",
       "4  2009-01-13           123"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "Bb66PeH3-U9l",
    "outputId": "e6c7b836-9a59-4d11-e8b6-eb32c44f2a23"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Blocks</th>\n",
       "      <th>Output_Satoshis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2009-01-09</td>\n",
       "      <td>14</td>\n",
       "      <td>700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2009-01-10</td>\n",
       "      <td>61</td>\n",
       "      <td>3050.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2009-01-11</td>\n",
       "      <td>93</td>\n",
       "      <td>4650.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2009-01-12</td>\n",
       "      <td>106</td>\n",
       "      <td>4879.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2009-01-13</td>\n",
       "      <td>123</td>\n",
       "      <td>6150.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date  Blocks  Output_Satoshis\n",
       "0  2009-01-09      14            700.0\n",
       "1  2009-01-10      61           3050.0\n",
       "2  2009-01-11      93           4650.0\n",
       "3  2009-01-12     106           4879.0\n",
       "4  2009-01-13     123           6150.0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "UIutjIkl-U9l",
    "outputId": "ed06c66e-5160-4f6e-941e-14d8f63af03d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Transactions</th>\n",
       "      <th>Blocks</th>\n",
       "      <th>Output_Satoshis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2009-01-09</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "      <td>700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2009-01-10</td>\n",
       "      <td>61</td>\n",
       "      <td>61</td>\n",
       "      <td>3050.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2009-01-11</td>\n",
       "      <td>93</td>\n",
       "      <td>93</td>\n",
       "      <td>4650.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2009-01-12</td>\n",
       "      <td>94</td>\n",
       "      <td>106</td>\n",
       "      <td>4879.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2009-01-13</td>\n",
       "      <td>123</td>\n",
       "      <td>123</td>\n",
       "      <td>6150.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date  Transactions  Blocks  Output_Satoshis\n",
       "0  2009-01-09            14      14            700.0\n",
       "1  2009-01-10            61      61           3050.0\n",
       "2  2009-01-11            93      93           4650.0\n",
       "3  2009-01-12            94     106           4879.0\n",
       "4  2009-01-13           123     123           6150.0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# merge the two dataframes\n",
    "result = pd.merge(df_1,\n",
    "                 df_2[['Date', 'Blocks', 'Output_Satoshis']],\n",
    "                 on='Date')\n",
    "result.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "nk-NMtsw-U9l",
    "outputId": "3eeff7fc-d7d3-4e9c-de97-6350fc99a679"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Transactions</th>\n",
       "      <th>Blocks</th>\n",
       "      <th>Output_Satoshis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>3532.000000</td>\n",
       "      <td>3.532000e+03</td>\n",
       "      <td>3.532000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>153.053228</td>\n",
       "      <td>2.623631e+05</td>\n",
       "      <td>1.358730e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>37.978411</td>\n",
       "      <td>2.799358e+05</td>\n",
       "      <td>2.193636e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>4.000000e+00</td>\n",
       "      <td>2.000000e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>138.000000</td>\n",
       "      <td>1.161975e+04</td>\n",
       "      <td>3.047457e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>153.000000</td>\n",
       "      <td>1.586145e+05</td>\n",
       "      <td>9.436964e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>169.000000</td>\n",
       "      <td>5.145980e+05</td>\n",
       "      <td>1.835461e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>692.000000</td>\n",
       "      <td>2.035035e+06</td>\n",
       "      <td>6.735430e+07</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Transactions        Blocks  Output_Satoshis\n",
       "count   3532.000000  3.532000e+03     3.532000e+03\n",
       "mean     153.053228  2.623631e+05     1.358730e+06\n",
       "std       37.978411  2.799358e+05     2.193636e+06\n",
       "min        2.000000  4.000000e+00     2.000000e+02\n",
       "25%      138.000000  1.161975e+04     3.047457e+05\n",
       "50%      153.000000  1.586145e+05     9.436964e+05\n",
       "75%      169.000000  5.145980e+05     1.835461e+06\n",
       "max      692.000000  2.035035e+06     6.735430e+07"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "eaiHITgY-U9m",
    "outputId": "6200f972-cfec-49aa-dd8b-dc873c959b17"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 34236 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 267,
       "width": 409
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.kdeplot(result['Blocks'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "fUu7hsn1-U9m",
    "outputId": "62ef3d03-4fa0-4b22-ad9e-f99faf691140"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 36884 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 244,
       "width": 429
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.kdeplot(result['Transactions'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "8Oa6G48V-U9n",
    "outputId": "46a70667-31aa-4326-b9c5-fa37d253da13"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 30044 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 267,
       "width": 409
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.kdeplot(result['Output_Satoshis'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "XM2VzO-O-U9n",
    "outputId": "419acb1d-bf9a-4170-b4b3-c95c977dbe4a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Transactions per day')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 224052 characters]\n",
      "text/plain": [
       "<Figure size 1440x648 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 570,
       "width": 1329
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "plt.style.use('ggplot')\n",
    "sns.set_context(\"notebook\", font_scale=1.5, rc={\"lines.linewidth\": 1.5})\n",
    "\n",
    "g = plt.subplots(figsize=(20,9))\n",
    "g = sns.lineplot(x='Date', y='Transactions', data=result, palette='Blues_d')\n",
    "plt.title('Transactions per day')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "UVdkiHb0-U9n",
    "outputId": "b5fec5cd-72b0-4761-85b0-bb7b07289277"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Blocks per day')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 186308 characters]\n",
      "text/plain": [
       "<Figure size 1440x648 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 570,
       "width": 1334
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "plt.style.use('ggplot')\n",
    "sns.set_context(\"notebook\", font_scale=1.5, rc={\"lines.linewidth\": 1.5})\n",
    "\n",
    "g = plt.subplots(figsize=(20,9))\n",
    "g = sns.lineplot(x='Date', y='Blocks', data=result, palette='Blues_d')\n",
    "plt.title('Blocks per day')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "LX2ltABE-U9n",
    "outputId": "4fa594b6-7545-4661-b309-4eb859979c06"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Sum of all satoshis spent each day')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 148788 characters]\n",
      "text/plain": [
       "<Figure size 1440x648 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 570,
       "width": 1309
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "g = plt.subplots(figsize=(20,9))\n",
    "g = sns.lineplot(x='Date', y='Output_Satoshis', data=result, palette='BuGn_r')\n",
    "plt.title('Sum of all satoshis spent each day')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "2X_TZjMu-U9o",
    "outputId": "64d1558a-cde7-43be-a812-06537976beb2"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.PairGrid at 0x165069320>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "[encoded data, 176968 characters]\n",
      "text/plain": [
       "<Figure size 540x540 with 12 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 551,
       "width": 547
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check the relation among the features of data\n",
    "sns.set(style=\"ticks\")\n",
    "sns.pairplot(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "CvTtBZiH-U9o",
    "outputId": "95b1d249-17fc-4bb5-809d-ec2f6f212429"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.61916492 -0.93730866 -3.66189407]\n",
      " [-0.61809349 -0.93714074 -2.42417364]\n",
      " [-0.617364   -0.93702641 -1.58147036]\n",
      " ...\n",
      " [-0.29173402  0.96100967 -0.08040515]\n",
      " [-0.29118499  0.83351596  0.26194305]\n",
      " [-0.37942782  0.12385633 -2.08182543]]\n"
     ]
    }
   ],
   "source": [
    "# select the three most important features (Transactions, Blocks, Output Satoshis) from the data\n",
    "data = result[['Output_Satoshis','Blocks','Transactions']]\n",
    "outliers_fraction=0.05\n",
    "scaler = StandardScaler()\n",
    "np_scaled = scaler.fit_transform(data)\n",
    "data = pd.DataFrame(np_scaled).to_numpy()\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PZ2vf0nY-U9o"
   },
   "source": [
    "## Example 1: DeepLog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "id": "7QMpoT-0-U9o",
    "outputId": "a498ffd9-df43-4dd7-f4e8-0859ec874e06",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "100/100 [==============================] - 16s 11ms/step - loss: 1.0956 - val_loss: 0.3173\n",
      "Epoch 2/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4987 - val_loss: 0.2470\n",
      "Epoch 3/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3694 - val_loss: 0.1844\n",
      "Epoch 4/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3769 - val_loss: 0.1988\n",
      "Epoch 5/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.5605 - val_loss: 0.1917\n",
      "Epoch 6/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.5632 - val_loss: 0.1911\n",
      "Epoch 7/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3737 - val_loss: 0.2046\n",
      "Epoch 8/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3943 - val_loss: 0.1861\n",
      "Epoch 9/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.3922 - val_loss: 0.1878\n",
      "Epoch 10/10\n",
      "100/100 [==============================] - 0s 2ms/step - loss: 0.4021 - val_loss: 0.2000\n"
     ]
    }
   ],
   "source": [
    "transformer_DL = DeepLogSKI()\n",
    "transformer_DL.fit(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "FFVMEusH-U9p",
    "outputId": "2d599283-e295-42c2-cc15-636fb0b5670d",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "prediction_labels_DL = transformer_DL.predict(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "0ZdcjuJ3-U9p",
    "outputId": "b8842503-07cf-4577-8a19-e14c9f05dad0"
   },
   "outputs": [],
   "source": [
    "prediction_score_DL = transformer_DL.predict_score(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "QcjVBJsv-U9p",
    "outputId": "9cc9feec-5f44-4e09-c72b-09122806a82e",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction Labels\n",
      " [[0]\n",
      " [0]\n",
      " [0]\n",
      " ...\n",
      " [0]\n",
      " [0]\n",
      " [1]]\n",
      "Prediction Score\n",
      " [[0.        ]\n",
      " [0.26495463]\n",
      " [0.22239052]\n",
      " ...\n",
      " [0.55641556]\n",
      " [0.48936596]\n",
      " [2.28045378]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Prediction Labels\\n\", prediction_labels_DL)\n",
    "print(\"Prediction Score\\n\", prediction_score_DL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "B6y-VhoA-U9p",
    "outputId": "6024cee9-2195-4a46-eb38-4bcb860b2c76"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3178\n",
       "1     354\n",
       "Name: anomaly_DeepLog, dtype: int64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a new column for storing the results of DeepLog method\n",
    "result['anomaly_DeepLog'] = pd.Series(prediction_labels_DL.flatten())\n",
    "result['anomaly_DeepLog'] = result['anomaly_DeepLog'].apply(lambda x: x == 1)\n",
    "result['anomaly_DeepLog'] = result['anomaly_DeepLog'].astype(int)\n",
    "result['anomaly_DeepLog'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "76MRjCtk-U9p",
    "outputId": "012428da-86b8-4891-844c-6dc46e37b072"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 143620 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 684
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "#anomaly\n",
    "a = result.loc[result['anomaly_DeepLog'] == 1]\n",
    "outlier_index=list(a.index)\n",
    "ax.plot(result['Transactions'], color='black', label = 'Normal', linewidth=1.5)\n",
    "ax.scatter(a.index ,a['Transactions'], color='red', label = 'Anomaly', s=16)\n",
    "ax.plot(pd.Series(prediction_score_DL.flatten()*10), color='blue', label = 'Score', linewidth=0.5)\n",
    "\n",
    "\n",
    "plt.legend()\n",
    "plt.title(\"Anamoly Detection Using DeepLog\")\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('Transactions')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "id": "UQkOGQWt-U9q",
    "outputId": "760eea07-80e9-457c-88c4-9a6bcda69aea"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 133460 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 681
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "#anomaly\n",
    "a = result.loc[result['anomaly_DeepLog'] == 1]\n",
    "ax.plot(result['Blocks'], color='black', label = 'Normal', linewidth=1.5)\n",
    "ax.scatter(a.index ,a['Blocks'], color='red', label = 'Anomaly', s=16)\n",
    "ax.plot(pd.Series(prediction_score_DL.flatten()*1e5), color='blue', label = 'Score', linewidth=0.5)\n",
    "\n",
    "\n",
    "plt.legend()\n",
    "plt.title(\"Anamoly Detection Using DeepLog\")\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('Blocks')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "id": "_ZTX7H7L-U9q",
    "outputId": "2301e3a8-8d7d-44f9-9097-0f9761535322"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 101796 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 671
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "#anomaly\n",
    "a = result.loc[result['anomaly_DeepLog'] == 1]\n",
    "ax.plot(result['Output_Satoshis'], color='black', label = 'Normal', linewidth=1.5)\n",
    "ax.scatter(a.index ,a['Output_Satoshis'], color='red', label = 'Anomaly', s=16)\n",
    "ax.plot(pd.Series(prediction_score_DL.flatten()*1e6), color='blue', label = 'Score', linewidth=0.5)\n",
    "\n",
    "\n",
    "plt.legend()\n",
    "plt.title(\"Anamoly Detection Using DeepLog\")\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('Output_Satoshis')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 286140 characters]\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 558,
       "width": 562
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure(figsize=(10,10))\n",
    "ax = fig.add_subplot(111, projection='3d')\n",
    "#ax.set_zlabel(\"x_composite_3\")\n",
    "ax.scatter(data[:, 0], data[:, 1], zs=data[:, 2], s=5, lw=1, label=\"inliers\",c=\"blue\")\n",
    "# Plot x's for the ground truth outliers\n",
    "ax.scatter(data[outlier_index,0],data[outlier_index,1], data[outlier_index,2],\n",
    "           lw=1, s=5, c=\"red\", label=\"outliers\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2vfa0ibf-U9q"
   },
   "source": [
    "## Example 2: Telemanom"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "N2RStdU_-U9q",
    "outputId": "77422b8f-6c24-4e7a-c477-2bf612f9061f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "43/43 [==============================] - 16s 148ms/step - loss: 1.3042 - val_loss: 0.9949\n"
     ]
    }
   ],
   "source": [
    "#regressor using historical data by using previous data to predict furture data\n",
    "transformer_TL = TelemanomSKI()\n",
    "transformer_TL.fit(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "id": "I3nvHUCb-U9q",
    "outputId": "5f8a61bf-7887-4943-b70e-6df03019f464"
   },
   "outputs": [],
   "source": [
    "prediction_labels_TL = transformer_TL.predict(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "id": "SplHaWJJ-U9r",
    "outputId": "edc47217-038a-4e11-b95b-bf885e677a58"
   },
   "outputs": [],
   "source": [
    "prediction_score_TL = transformer_TL.predict_score(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "id": "5mYPYtGA-U9r",
    "outputId": "fd17642a-c12b-4dd2-d375-bf5070232764"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction Labels\n",
      " [[0]\n",
      " [0]\n",
      " [0]\n",
      " ...\n",
      " [0]\n",
      " [0]\n",
      " [0]]\n",
      "Prediction Score\n",
      " [[25.52807608]\n",
      " [25.40674529]\n",
      " [25.26002474]\n",
      " ...\n",
      " [18.68037199]\n",
      " [18.66470849]\n",
      " [18.646084  ]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Prediction Labels\\n\", prediction_labels_TL)\n",
    "print(\"Prediction Score\\n\", prediction_score_TL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "NC2EYAbV-U9r",
    "outputId": "1cbd7eb4-ce20-43b0-c086-7a58b27a773e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3189\n",
       "1     343\n",
       "Name: anomaly_Telemanom, dtype: int64"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a new column for storing the results of Telemanom method\n",
    "result['anomaly_Telemanom'] = pd.Series(prediction_labels_TL.flatten()) #somehow make into 1d\n",
    "result['anomaly_Telemanom'] = result['anomaly_Telemanom'].apply(lambda x: x == 1)\n",
    "result['anomaly_Telemanom'] = result['anomaly_Telemanom'].astype(int)\n",
    "result['anomaly_Telemanom'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "id": "-HBPP9RT-U9r",
    "outputId": "6d0b93d3-61bc-4379-d048-e0bbd8a5bd0a"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 135480 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 684
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "#anomaly\n",
    "a = result.loc[result['anomaly_Telemanom'] == 1]\n",
    "outlier_index=list(a.index)\n",
    "ax.plot(result['Transactions'], color='black', label = 'Normal', linewidth=1.5)\n",
    "ax.scatter(a.index ,a['Transactions'], color='red', label = 'Anomaly', s=16)\n",
    "ax.plot(pd.Series(prediction_score_TL.flatten()*10), color='blue', label = 'Score', linewidth=1)\n",
    "plt.legend()\n",
    "plt.title(\"Anamoly Detection Using Telemanom\")\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('Transactions')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "id": "Cpvf08eP-U9r",
    "outputId": "dc1dae6e-4b8b-4b33-c727-fdbb4cda1643"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 116440 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 681
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "#anomaly\n",
    "a = result.loc[result['anomaly_Telemanom'] == 1]\n",
    "ax.plot(result['Blocks'], color='black', label = 'Normal', linewidth=1.5)\n",
    "ax.scatter(a.index ,a['Blocks'], color='red', label = 'Anomaly', s=16)\n",
    "ax.plot(pd.Series(prediction_score_TL.flatten()*1e5), color='blue', label = 'Score', linewidth=1)\n",
    "plt.legend()\n",
    "plt.title(\"Anamoly Detection Using Telemanom\")\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('Blocks')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0       25.528076\n",
      "1       25.406745\n",
      "2       25.260025\n",
      "3       25.198927\n",
      "4       25.134838\n",
      "          ...    \n",
      "3417    18.710588\n",
      "3418    18.694653\n",
      "3419    18.680372\n",
      "3420    18.664708\n",
      "3421    18.646084\n",
      "Length: 3422, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(pd.Series(prediction_score_TL.flatten()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "id": "hC2jpBrZ-U9s",
    "outputId": "60f16bb0-a3ce-4ab4-bb37-f208b2c18706",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 102528 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 390,
       "width": 671
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,6))\n",
    "\n",
    "#anomaly\n",
    "a = result.loc[result['anomaly_Telemanom'] == 1]\n",
    "ax.plot(result['Output_Satoshis'], color='black', label = 'Normal', linewidth=1.5)\n",
    "ax.scatter(a.index ,a['Output_Satoshis'], color='red', label = 'Anomaly', s=16)\n",
    "ax.plot(pd.Series(prediction_score_TL.flatten()*1e6), color='blue', label = 'Score', linewidth=1)\n",
    "plt.legend()\n",
    "plt.title(\"Anamoly Detection Using Telemanom\")\n",
    "plt.xlabel('Date')\n",
    "plt.ylabel('Output_Satoshis')\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 270688 characters]\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 558,
       "width": 562
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure(figsize=(10,10))\n",
    "ax = fig.add_subplot(111, projection='3d')\n",
    "#ax.set_zlabel(\"x_composite_3\")\n",
    "ax.scatter(data[:, 0], data[:, 1], zs=data[:, 2], s=4, lw=1, label=\"inliers\",c=\"blue\")\n",
    "# Plot x's for the ground truth outliers\n",
    "ax.scatter(data[outlier_index,0],data[outlier_index,1], data[outlier_index,2],\n",
    "           lw=2, s=30, c=\"red\", label=\"outliers\")\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "evax-5IF-U9s"
   },
   "source": [
    "## Searcher Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "id": "qG2gYe9H-U9s",
    "outputId": "7bd27e67-0bc2-4d6f-9cac-15c9cf97b3a5"
   },
   "outputs": [],
   "source": [
    "from d3m import index\n",
    "from d3m.metadata.base import ArgumentType\n",
    "from d3m.metadata.pipeline import Pipeline, PrimitiveStep\n",
    "from axolotl.backend.simple import SimpleRunner\n",
    "from tods import generate_dataset, generate_problem\n",
    "from tods.searcher import BruteForceSearch\n",
    "from tods import generate_dataset, load_pipeline, evaluate_pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "id": "J_WmjBBC-U9s"
   },
   "outputs": [],
   "source": [
    "table_path = 'nodateblock.csv'\n",
    "target_index = 3 # what column is the target\n",
    "time_limit = 30 # How many seconds you wanna search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "id": "0txdW5de-U9t"
   },
   "outputs": [],
   "source": [
    "metric = 'F1_MACRO' # F1 on both label 0 and 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "id": "8VowONRq-U9t"
   },
   "outputs": [],
   "source": [
    "# Read data and generate dataset and problem\n",
    "df = pd.read_csv(table_path)\n",
    "dataset = generate_dataset(df, target_index=target_index)\n",
    "problem_description = generate_problem(dataset, metric)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "id": "BIg-SdEw-U9t"
   },
   "outputs": [],
   "source": [
    "# Start backend\n",
    "backend = SimpleRunner(random_seed=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "id": "a42_XF4v-T_E"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "While loading primitive 'tods.data_processing.dataset_to_dataframe', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.data_processing.dataset_to_dataframe' without checking requirements.\n",
      "While loading primitive 'tods.data_processing.column_parser', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.data_processing.column_parser' without checking requirements.\n",
      "While loading primitive 'tods.data_processing.extract_columns_by_semantic_types', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.data_processing.extract_columns_by_semantic_types' without checking requirements.\n",
      "While loading primitive 'tods.timeseries_processing.transformation.axiswise_scaler', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.timeseries_processing.transformation.axiswise_scaler' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_mean', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_mean' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_ae', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_ae' without checking requirements.\n",
      "While loading primitive 'tods.data_processing.construct_predictions', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.data_processing.construct_predictions' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_vae', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_vae' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_cof', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_cof' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_sod', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_sod' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_abod', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_abod' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_hbos', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_hbos' without checking requirements.\n",
      "While loading primitive 'tods.detection_algorithm.pyod_iforest', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.detection_algorithm.pyod_iforest' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_median', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_median' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_g_mean', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_g_mean' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_abs_energy', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_abs_energy' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_abs_sum', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_abs_sum' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_h_mean', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_h_mean' without checking requirements.\n",
      "While loading primitive 'tods.feature_analysis.statistical_maximum', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.feature_analysis.statistical_maximum' without checking requirements.\n",
      "While loading primitive 'tods.timeseries_processing.transformation.standard_scaler', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.timeseries_processing.transformation.standard_scaler' without checking requirements.\n",
      "While loading primitive 'tods.timeseries_processing.transformation.power_transformer', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.timeseries_processing.transformation.power_transformer' without checking requirements.\n",
      "While loading primitive 'tods.timeseries_processing.transformation.quantile_transformer', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.timeseries_processing.transformation.quantile_transformer' without checking requirements.\n",
      "While loading primitive 'tods.timeseries_processing.transformation.moving_average_transform', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.timeseries_processing.transformation.moving_average_transform' without checking requirements.\n",
      "While loading primitive 'tods.timeseries_processing.transformation.simple_exponential_smoothing', an error has been detected: (grpcio 1.36.1 (/Users/wangyanghe/anaconda3/envs/tods2/lib/python3.6/site-packages), Requirement.parse('grpcio~=1.34.0'), {'tensorflow'})\n",
      "Attempting to load primitive 'tods.timeseries_processing.transformation.simple_exponential_smoothing' without checking requirements.\n"
     ]
    }
   ],
   "source": [
    "# Start search algorithm\n",
    "search = BruteForceSearch(problem_description=problem_description,\n",
    "                          backend=backend)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "GLVMi4pg-T_F",
    "outputId": "a86d614b-ef69-4502-a73d-a234e03864ca"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout (Dropout)            (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_4 (Dropout)          (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 4ms/step - loss: 1.3996 - val_loss: 1.1556\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/Users/wangyanghe/Desktop/Research/tods/tods/searcher/brute_force_search.py\", line 62, in _search\n",
      "    for error in pipeline_result.error:\n",
      "TypeError: 'NoneType' object is not iterable\n",
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_6 (Dense)              (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_5 (Dropout)          (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_6 (Dropout)          (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_9 (Dense)              (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_10 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_9 (Dropout)          (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_11 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 3ms/step - loss: 1.5395 - val_loss: 1.3249\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/Users/wangyanghe/Desktop/Research/tods/tods/searcher/brute_force_search.py\", line 62, in _search\n",
      "    for error in pipeline_result.error:\n",
      "TypeError: 'NoneType' object is not iterable\n",
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_12 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_10 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_13 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_11 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_14 (Dense)             (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_12 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_15 (Dense)             (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_13 (Dropout)         (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_16 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_14 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_17 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 3ms/step - loss: 1.8423 - val_loss: 1.4715\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/Users/wangyanghe/Desktop/Research/tods/tods/searcher/brute_force_search.py\", line 62, in _search\n",
      "    for error in pipeline_result.error:\n",
      "TypeError: 'NoneType' object is not iterable\n",
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_3\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_18 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_15 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_19 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_16 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_20 (Dense)             (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_17 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_21 (Dense)             (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_18 (Dropout)         (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_22 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_19 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_23 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 3ms/step - loss: 1.4077 - val_loss: 1.4583\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/Users/wangyanghe/Desktop/Research/tods/tods/searcher/brute_force_search.py\", line 62, in _search\n",
      "    for error in pipeline_result.error:\n",
      "TypeError: 'NoneType' object is not iterable\n",
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_4\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_24 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_20 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_25 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_21 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_26 (Dense)             (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_22 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_27 (Dense)             (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_23 (Dropout)         (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_28 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_24 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_29 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 3ms/step - loss: 1.3654 - val_loss: 1.1456\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/Users/wangyanghe/Desktop/Research/tods/tods/searcher/brute_force_search.py\", line 62, in _search\n",
      "    for error in pipeline_result.error:\n",
      "TypeError: 'NoneType' object is not iterable\n",
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_5\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_30 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_25 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_31 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_26 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_32 (Dense)             (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_27 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_33 (Dense)             (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_28 (Dropout)         (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_34 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_29 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_35 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 3ms/step - loss: 1.4800 - val_loss: 1.5099\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Traceback (most recent call last):\n",
      "  File \"/Users/wangyanghe/Desktop/Research/tods/tods/searcher/brute_force_search.py\", line 62, in _search\n",
      "    for error in pipeline_result.error:\n",
      "TypeError: 'NoneType' object is not iterable\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_6\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_36 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_30 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_37 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_31 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_38 (Dense)             (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_32 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_39 (Dense)             (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_33 (Dropout)         (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_40 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_34 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_41 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 4ms/step - loss: 1.4285 - val_loss: 1.2025\n"
     ]
    }
   ],
   "source": [
    "# Find the best pipeline\n",
    "best_runtime, best_pipeline_result = search.search_fit(input_data=[dataset], time_limit=time_limit)\n",
    "# print(best_runtime)\n",
    "best_pipeline = best_runtime.pipeline\n",
    "best_output = best_pipeline_result.output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "id": "31nHAeqc-T_F",
    "outputId": "3131a52a-1a9d-41fd-8484-0a0dae1dc3e4"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Not all provided hyper-parameters for the data preparation pipeline 79ce71bd-db96-494b-a455-14f2e2ac5040 were used: ['method', 'number_of_folds', 'randomSeed', 'shuffle', 'stratified']\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_7\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_42 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_35 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_43 (Dense)             (None, 6)                 42        \n",
      "_________________________________________________________________\n",
      "dropout_36 (Dropout)         (None, 6)                 0         \n",
      "_________________________________________________________________\n",
      "dense_44 (Dense)             (None, 1)                 7         \n",
      "_________________________________________________________________\n",
      "dropout_37 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_45 (Dense)             (None, 4)                 8         \n",
      "_________________________________________________________________\n",
      "dropout_38 (Dropout)         (None, 4)                 0         \n",
      "_________________________________________________________________\n",
      "dense_46 (Dense)             (None, 1)                 5         \n",
      "_________________________________________________________________\n",
      "dropout_39 (Dropout)         (None, 1)                 0         \n",
      "_________________________________________________________________\n",
      "dense_47 (Dense)             (None, 6)                 12        \n",
      "=================================================================\n",
      "Total params: 116\n",
      "Trainable params: 116\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "69/69 [==============================] - 1s 3ms/step - loss: 1.5863 - val_loss: 2.3615\n"
     ]
    }
   ],
   "source": [
    "# Evaluate the best pipeline\n",
    "best_scores = search.evaluate(best_pipeline).scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "id": "E9xPN5L1-T_F",
    "outputId": "1fa638b1-aad0-42df-a142-74163d7c7bd9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Search History:\n",
      "----------------------------------------------------\n",
      "Pipeline id: eb0ae5a5-b8a5-44cb-9162-f8460ef909c4\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.571701    0.571701           0     0\n",
      "----------------------------------------------------\n",
      "Pipeline id: 50571e7f-5d1a-4c3b-a216-ee041ad9f4c9\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.534343    0.534343           0     0\n",
      "----------------------------------------------------\n",
      "Pipeline id: 99639330-803d-43c8-9bb2-666117fbf87a\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.503497    0.503497           0     0\n",
      "----------------------------------------------------\n",
      "Pipeline id: 043e2d74-feea-4bef-8a94-c9351d538ed3\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.493586    0.493586           0     0\n",
      "----------------------------------------------------\n",
      "Pipeline id: c3371b86-2e5b-4531-9448-538483e6fec1\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.481985    0.481985           0     0\n",
      "----------------------------------------------------\n",
      "Pipeline id: 2555ca5f-11bd-4d78-9e4d-da80194984f3\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.465044    0.465044           0     0\n"
     ]
    }
   ],
   "source": [
    "print('Search History:')\n",
    "for pipeline_result in search.history:\n",
    "    print('-' * 52)\n",
    "    print('Pipeline id:', pipeline_result.pipeline.id)\n",
    "    print(pipeline_result.scores)\n",
    "    \n",
    "#leader board of pipelines with performance scores, the pipeline id could be used to access all these pipelines for different uses, for example higher recall"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "id": "hwzKSMDJ-T_F",
    "outputId": "80915d93-d46b-4471-fed6-b11001980f68"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best pipeline:\n",
      "----------------------------------------------------\n",
      "Pipeline id: eb0ae5a5-b8a5-44cb-9162-f8460ef909c4\n",
      "Pipeline json: {\"id\": \"eb0ae5a5-b8a5-44cb-9162-f8460ef909c4\", \"schema\": \"https://metadata.datadrivendiscovery.org/schemas/v0/pipeline.json\", \"created\": \"2021-12-03T22:14:35.381223Z\", \"inputs\": [{\"name\": \"inputs\"}], \"outputs\": [{\"data\": \"steps.7.produce\", \"name\": \"output predictions\"}], \"steps\": [{\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"c78138d9-9377-31dc-aee8-83d9df049c60\", \"version\": \"0.3.0\", \"python_path\": \"d3m.primitives.tods.data_processing.dataset_to_dataframe\", \"name\": \"Extract a DataFrame from a Dataset\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"inputs.0\"}}, \"outputs\": [{\"id\": \"produce\"}]}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"81235c29-aeb9-3828-911a-1b25319b6998\", \"version\": \"0.6.0\", \"python_path\": \"d3m.primitives.tods.data_processing.column_parser\", \"name\": \"Parses strings into their types\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.0.produce\"}}, \"outputs\": [{\"id\": \"produce\"}]}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"a996cd89-ddf0-367f-8e7f-8c013cbc2891\", \"version\": \"0.4.0\", \"python_path\": \"d3m.primitives.tods.data_processing.extract_columns_by_semantic_types\", \"name\": \"Extracts columns by semantic type\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.1.produce\"}}, \"outputs\": [{\"id\": \"produce\"}], \"hyperparams\": {\"semantic_types\": {\"type\": \"VALUE\", \"data\": [\"https://metadata.datadrivendiscovery.org/types/Attribute\"]}}}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"a996cd89-ddf0-367f-8e7f-8c013cbc2891\", \"version\": \"0.4.0\", \"python_path\": \"d3m.primitives.tods.data_processing.extract_columns_by_semantic_types\", \"name\": \"Extracts columns by semantic type\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.0.produce\"}}, \"outputs\": [{\"id\": \"produce\"}], \"hyperparams\": {\"semantic_types\": {\"type\": \"VALUE\", \"data\": [\"https://metadata.datadrivendiscovery.org/types/TrueTarget\"]}}}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"642de2e7-5590-3cab-9266-2a53c326c461\", \"version\": \"0.0.1\", \"python_path\": \"d3m.primitives.tods.timeseries_processing.transformation.axiswise_scaler\", \"name\": \"Axis_wise_scale\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.2.produce\"}}, \"outputs\": [{\"id\": \"produce\"}]}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"30bc7cec-2ccc-34bc-9df8-2095bf3b1ae2\", \"version\": \"0.1.0\", \"python_path\": \"d3m.primitives.tods.feature_analysis.statistical_mean\", \"name\": \"Time Series Decompostional\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.4.produce\"}}, \"outputs\": [{\"id\": \"produce\"}]}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"67e7fcdf-d645-3417-9aa4-85cd369487d9\", \"version\": \"0.0.1\", \"python_path\": \"d3m.primitives.tods.detection_algorithm.pyod_ae\", \"name\": \"TODS.anomaly_detection_primitives.AutoEncoder\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.5.produce\"}}, \"outputs\": [{\"id\": \"produce\"}], \"hyperparams\": {\"contamination\": {\"type\": \"VALUE\", \"data\": 0.01}}}, {\"type\": \"PRIMITIVE\", \"primitive\": {\"id\": \"2530840a-07d4-3874-b7d8-9eb5e4ae2bf3\", \"version\": \"0.3.0\", \"python_path\": \"d3m.primitives.tods.data_processing.construct_predictions\", \"name\": \"Construct pipeline predictions output\"}, \"arguments\": {\"inputs\": {\"type\": \"CONTAINER\", \"data\": \"steps.6.produce\"}, \"reference\": {\"type\": \"CONTAINER\", \"data\": \"steps.1.produce\"}}, \"outputs\": [{\"id\": \"produce\"}]}], \"digest\": \"ef56a2eacfb9b8d0a3680124dd0aae329c909ac4051e4d621d7fa8b0a130912d\"}\n",
      "Output:\n",
      "     d3mIndex  anomaly\n",
      "0           0        0\n",
      "1           1        0\n",
      "2           2        0\n",
      "3           3        0\n",
      "4           4        0\n",
      "...       ...      ...\n",
      "2432     2432        0\n",
      "2433     2433        0\n",
      "2434     2434        0\n",
      "2435     2435        0\n",
      "2436     2436        1\n",
      "\n",
      "[2437 rows x 2 columns]\n",
      "Scores:\n",
      "     metric     value  normalized  randomSeed  fold\n",
      "0  F1_MACRO  0.571701    0.571701           0     0\n"
     ]
    }
   ],
   "source": [
    "print('Best pipeline:')\n",
    "print('-' * 52)\n",
    "print('Pipeline id:', best_pipeline.id)\n",
    "print('Pipeline json:', best_pipeline.to_json())\n",
    "print('Output:')\n",
    "print(best_output)\n",
    "print('Scores:')\n",
    "print(best_scores)\n",
    "#best pipeline outputs the id and structrue of json file, it can be re-loaded by tods and to be shared to other systems"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Blockchain.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
