How to set up an environment to train and analyze PCAP data on a Ubuntu machine in this guide version 20.04 (Focal Fossa) is used.

1. Install Anaconda

• To download and install Anaconda on Ubuntu via the command line, follow these steps:

  1. Update your package list and install dependencies:

     sudo apt update sudo apt install wget

  2. Download the Anaconda installer:

     Visit the Anaconda distribution page to get the link to the latest version. As of now, you can use the following command to download Anaconda:

     wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

     Note: Check the website for the latest version and update the URL accordingly

  3. Install

     chmod +x Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh -b -p $HOME/miniconda3

  4. Initialize Miniconda:
     If the installer does not automatically initialize Miniconda, you can do it manually:

     This command configures your shell to use Conda. You may need to restart your terminal or run source ~/.bashrc for the changes to take effect.

  5. Verify and Update

2. Create a Python and Developing Environment:

• Run the following command to create a new conda environment with Python and JupyterLab:

• Run the following command to refresh the shell and activate the conda environment:

• With the conda environment activated, run the following command to install JupyterLab:

• With the conda environment activated, run the following command to start JupyterLab:

• This command will open JupyterLab in your default web browser.

3. Install NVIDIA Python package

• With the conda environment still activated, install the NVIDIA Python package using pip:

Direct install if conda installation of CUDA or cuDNN does not work

- Install NVIDIA CUDA Toolkit

• NVIDIA CUDA Toolkit - Free Tools and Training .

  1. Add the NVIDIA package repository:

     First, add the repository to your system:

  2. Add the CUDA repository public key:

  3. Add the CUDA repository:

  4. Update the package list and install the CUDA Toolkit:

  5. Set up environment variables:

     Add CUDA to your PATH and LD_LIBRARY_PATH. You can do this by editing your .bashrc or .zshrc file:

     Add the following lines at the end of the file:

     Replace <version> with the version of CUDA you installed, such as 11.8.

     Save the file and reload it:

  6. Verify the installation:

     You can check if CUDA is installed correctly by running:

     You should see the version of CUDA installed.

- Install NVIDIA cuDNN

• To install cuDNN (CUDA Deep Neural Network library) on Ubuntu, follow these steps:

  1. Download cuDNN:

     Go to the NVIDIA cuDNN download page and sign in with your NVIDIA Developer account (create one if you don’t have it). Select the version of cuDNN that matches your CUDA version and download the appropriate tarball file for Linux:

  2. Extract the cuDNN tarball:

     Once downloaded, extract the files. For example, if you downloaded cudnn-xx.x-linux-x64-vx.x.x.tgz:

     Replace xx.x and vx.x.x with the specific version numbers of the file you downloaded.

  3. Copy cuDNN files to CUDA directory:

     Assuming you have CUDA installed in /usr/local/cuda, you need to copy the cuDNN files to the appropriate directories:

  4. Set permissions:

     To ensure that the libraries are accessible, set the appropriate permissions:

  5. Verify the installation:

     Check that the cuDNN library is correctly installed and linked:

     This should output the version of cuDNN installed.

- Install LibPCAP and Wireshark for Capture

• LibPCAP and Wireshark are necessary to perform packet capture and packet decoding.

• It is also optional to use tcpdump to do packet capture as well

- Download, Compile and Install Argus and RA

• Argus is a network flow monitor that will generate the features for the machine learning dataset

• After Argus is installed

- Start Packet Capture on Endpoints of Interest

• Go to endpoints that have Wireshark or tcpdump installed and run a capture either on a wireless interface or cabled ethernet interface

• Pick either tcpdump or tshark methods to capture packets

• Run the capture passively on the device using tcpdump

• Run capture using tshark (Wireshark toolkit)

• Replace <interface> with the actual device name from ‘ip -s link’ or ‘ip addr show’

• Repeat on all the endpoints of interest

• Wait for a couple of hours or a couple of days to gather packets, the more the better

- Generate CSV from Capture Files

• Using Argus and RA create a CSV file that is digestable by the machine learning code

- Copy Anomaly Detection Notebook

• Copy the Anomaly Detection Notebook onto local Jupyter Notebook

Anomaly Detection Notebook - Kaggle

• Change the input file in the python code to use traffic.csv instead of the default csv file.

• Add the full path of the CSV file is necessary

- Run the code and the get the results using Jupyter Notebook

- (Optional) Use NTFLlowLyzer to get the features from PCAP data

GitHub - ahlashkari/NTLFlowLyzer

- (Optional) Install OpenWebUI to Create a GUI for the Output