TinkerDad

Installing YETI with Portainer on Ubuntu 22.04

2022-10-17T14:23:00.004-07:00

Another day, another project. I have been playing with some open-source Intelligence Platforms (I installed OpenCTI, and MISP recently with Portainer). I recently found another project called YETI 'Your Everyday Threat Intelligence'

For background, I already have a VM setup running Portainer (It is probably overworked, but it's only for testing, so not too concerned about overloading). One change on the VM was to create a folder:

/tmp/docker-yeti-exports

In hindsight I would have changed the location in the docker-compose but missed it (needs to be rebuit upon reboot).

The setup of Yeti inside of Portainer took a little more than the previous builds as it could not build the image from the docker-compose. I am new to this, so not entirely sure, but I think they don't host the image on GitHub or docker for it to build from.

I had to download a few files from the Yeti GitHub

requirements.txt
dockerfile
docker-entrypoint.sh

Because I will be updating the requirements.txt and I had issues with the original docker-entrypoint I had added updated the Dockerfile with the following:

RUN git clone https://github.com/yeti-platform/yeti.git /opt/yeti;
COPY requirements.txt /opt/yeti
COPY docker-entrypoint.sh /docker-entrypoint.sh

In the requirements.txt, I added a new application and added forced version. This is due to an issue with flask and werkzeug

flask=2.1.2
werkzeug=2.12

Next, I created a .tar file with those three files. Those files were included in the tar because Portainer will consider the files part of the default path (not needing to include local paths in your script). I found that information somewhere else (I think it was on Reddit). The tar file was used to create an image called yeti:latest as shown below:

The image took a few minutes to create, after that it was time to add a new stack to Portainer.

First things first, I had to update the docker-compose for the new image. I changed out yeti1:master to yeti:latest as below

version: '3.3'
services:

  yeti:
    image: yeti:latest
    ports:
      - "5000:5000"
    command: ['webserver']
    depends_on:
      - redis
      - mongodb
    volumes:
      - /tmp/docker-yeti-exports:/opt/yeti/exports

  feeds:
    image: yeti:latest
    command: ['feeds']
    depends_on:
      - redis
      - mongodb
      - yeti
    environment:
      - TLDEXTRACT_CACHE=/tmp/tldextract.cache

  analytics:
    image: yeti:latest
    command: ['analytics']
    depends_on:
      - redis
      - mongodb
      - yeti
    environment:
      - TLDEXTRACT_CACHE=/tmp/tldextract.cache

  beat:
    image: yeti:latest
    command: ['beat']
    depends_on:
      - redis
      - mongodb
      - yeti
      - feeds
      - analytics
      - exports

  exports:
    image: yeti:latest
    command: ['exports']
    depends_on:
      - redis
      - mongodb
      - yeti
    volumes:
      - /tmp/docker-yeti-exports:/opt/yeti/exports

  oneshot:
    image: yeti:latest
    command: ['oneshot']
    depends_on:
      - redis
      - mongodb
      - yeti

  redis:
    image: redis:latest

  mongodb:
    image: mongo:4.0.12
    environment:
      - MONGO_LOG_DIR=/dev/null
    command: mongod

I created a new stack (called yeti) and deployed it.

And the screen for YETI (Which I noticed did not have a login screen). Not sure if that is normal btw as it's my first time using it.

Building Vulnerability Scanners with Portainer

2022-10-16T09:34:00.009-07:00

Currently, I am in school for my Master, and we had an assignment to conduct vulnerability scanning on our home network. It has been a while since I installed Nessus or OpenVAS, and technology has certainly changed.

I have been using Portainer recently for most of my Docker containers and wanted to see if it was that easy for Nessus or OpenVAS.

For Nessus, I did a search for 'Nessus docker-compose'

version: '3.1'

services:

  nessus:
    image: tenableofficial/nessus
    restart: always
    container_name: nessus
    environment:
      USERNAME: <user>
      PASSWORD: <password>
      ACTIVATION_CODE: <code>
    ports:
      - 8834:8834

I changed the username/password and activation code. Then I went into Portainer, created a new stack, and placed the above in the web editor.

From there, I clicked deploy stack. About 20 minutes later (plugin updates on Nessus), I was up and operational on Nessus Essentials. One side note to this is that Essentials will only scan 16 IPs, but it's free.

For OpenVAS I searched on Google for 'OpenVAS docker-compose' and found https://github.com/immauss/openvas. From there, I used the below:

version: "3"
services:
  openvas:
    ports:
      - "8080:9392"
    environment:
      - "PASSWORD=admin"
      - "USERNAME=admin"
      - "RELAYHOST=172.17.0.1"
      - "SMTPPORT=25"
      - "REDISDBS=512" # number of Redis DBs to use
      - "QUIET=false"  # dump feed sync noise to /dev/null
      - "NEWDB=false"  # only use this for creating a blank DB 
      - "SKIPSYNC=true" # Skips the feed sync on startup.
      - "RESTORE=false"  # This probably not be used from compose... see docs.
      - "DEBUG=false"  # This will cause the container to stop and not actually start gvmd
      - "HTTPS=false"  # wether to use HTTPS or not
    volumes:
      - "openvas:/data"
    container_name: openvas
    image: immauss/openvas:$TAG
volumes:
  openvas:

Same procedures as Nessus. Opened Portainer, and added new stack. The web editor copied the above information and deployed stack. On this one, I forgot to update the username/password for my instance. So that shows as a vulnerability as you conduct a scan.

Overall, both of these installs were very easy, and I was up and running in about 30 minutes and running scans against my home network.

Installing MISP with Portainer on Unbuntu 22.04 VM

2022-10-08T23:13:00.000-07:00

I am installing MISP on the same VM that I have running OpenCTI. As Portainer is already installed on there.

I chose Coolacid's docker buildout

First things first you have to build out a directory structure on the host VM.

sudo mkdir /data/compose/#/

Additional folder under the number (mine was 2) are:

files
ssl
server-configs
logs

Back at Portainers web ui. Select Stack from the Left menu, and click +Add Stack

Next name the stack (lower case) and use the web editor upload the docker-compose

Click Deploy Stack at the bottom of the page and you are ready to access the MISP login screen.

Default email: admin@admin.test
password: admin

Password will be required to be changed on first login

Installing OpenCTI with Portainer on Ubuntu 22.04

2022-10-02T10:46:00.002-07:00

Having played around with SecurityOnion I was starting to look into Threat/Intel feeds, which lead me to a few applications: OpenCTI, and MISP to name a few. Today I am going to look at setting up a Docker instance of OpenCTI on Ubuntu 22.04 VM.

While researching OpenCTI I found documentation of setting up OpenCTI with Portainer. Having never heard of Portainer, I first wanted to see what that was all about.

From the website for Portainer: Container Management made easy. Sold! I have used Docker a few times, but mostly basic stuff like setting up a container, inspecting the container, ect. So I don't really have much experience, but from the looks of Portainer, it has a GUI front end and works with Docker and Kubernetes. I figured I could use it as I was going to use this system later to install a Docker instance of MISP on the same machine.

The basis of the install procedures came from here.

I had selected "Docker" option while installing Ubuntu 20.24 server. So I skipped the first part, and started with creating a swarm (On one computer mind)

docker swarm init --advertise-addr 192.168.1.100

This will setup a Docker swarm and my machine is the Manager node.

Installing Portainer

Below are the commands I ran on my Ubuntu VM for initial setup of Portainer.

mkdir -p /opt/portainer
cd /opt/portainer
curl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.yml

I updated the Ports associated in the portainer-agent-stack.yml (due to a conflict with OpenCTI)

ports:

-19000:9000

-18000:8000

Last step is deploy the Docker container

docker stack deploy --compose-file=portainer-agent-stack.yml portainer

Access Portainer from <UbuntuVM_IP>:1900

Installing OpenCTI

OpenCTI will be installed from within Portainer. A docker-compose file is required for the installation.

This file can be found at: https://netmanageitbackup.com:8083/url/kcyuuqdcuyk7ufks

This version had connectors setup for OTX, greynoise, abuseip, shodan, inetzer, and a few others. A few configuration are required with the above file, for instance, you will need to update all the UUIDs and add in your API from the above sites. Lastly, make sure you add your email address/password into the file in the below section:

- APP__ADMIN__EMAIL=

- APP__ADMIN__PASSWORD=

When logged into Portainer you create a new stack as shown below:

Next you provide a name and copy the docker file into the web editor as show below:

Lastly deploy the stack and wait about 30 minutes for it to fully build. Once complete you will be able to access the site at https://ip:8080.

Critical Path Security: SecurityOnion@Home

2022-07-01T16:51:00.000-07:00

Recently I have been trying to learn more about other types of information you can get from Zeek/Suricata (IP Reputation/DNS Rep), which previously lead me to add IOCs to Suricata with Datasets.

Today I am adding CriticalPathSecurity Threat Intel to Zeek on Security Onion 2.3.130. Overall it was a pretty simple install, and only really required one file edit (Salt file).

Following these steps

Clone the Critical Path Security Intelligence Feeds:

git clone https://github.com/CriticalPathSecurity/Zeek-Intelligence-Feeds.git /opt/so/saltstack/local/zeek/policy/intel/Zeek-Intelligence-Feeds

Copy the __load.zeek__ from default to local

cp /opt/so/saltstack/default/zeek/policy/intel/ /opt/so/saltstack/local/zeek/policy/intel/

Edit __load.zeek__

(Added @load integration/collective-intel & file instead of using one intel.dat, I added each file separately under the folder that Salt/Docker matches out on the host machine)

Update Salt

salt systemname_standalone system.highstate

__load.zeek__

Lets check the Intel Dashboards under Security Onion 2.3.130

Intel Dashboard Security Onion 2.3.130

The first IP address listed here was from abuse.ch and I did a nslookup for it to appear in the list.

Suricata: DataSets for IOCs

2022-06-28T00:38:00.001-07:00

After reviewing MalTrail, I wanted to see if there were other ways to provide the same/close to the same type of information based on software I was already using. This led me to Suricata and IPREP, and DATASETS.

While reading up, I found an article over at IDSTower about Datasets, and figured this would be a good starting point for comparing the two applications.

The article was setting up Datasets for bad domains, and based on the instructions it would be an easy add to Security Onion without having to really mess with any of the Salt files (I believe IPREP I will have to make some changes to Salt files, but that will be another article.)

Overall the process was pretty simple, I did change one thing, instead of adding iocs.rules, I added my rule to local.rules, and ran so-rule-update. I believe to add the iocs.rules as a separate source I will have to edit the Salt file for the IDSTools Docker.

After I had set up the new rule, I did a nslookup to a bad site: (From the Alerts Pane).

I think I will have to make a custom alert to see more information on these, or maybe a dashboard might work.

I would like to see the name of the bad IP/DNS entry, and possibly the country/region for the IP/DNS for a quick alert view pane.

Now MalTrail looks like: (screen capture taken from their Demo)

I would think to get the "info" section I would need to break out the DataSets per type of IOC as the Alert Description, the other ones are pretty standard fields I could pull.

Next, I think I will try and pull the MalTrail data into Logstash in SecurityOnion, think I read somewher e there is a Logstash setting in MalTrail, or it should be pretty easy to use the data file created from MalTrail to ingest into Logstash.

Malcom NSM - Installation

2022-06-26T23:37:00.003-07:00

Using the directions from their Github, I set out tonight to set up Malcolm NSM. As I was running out of space on my machine (and truthfully too lazy to open it up and add more drives, I purchased an NVME M2 external USB 3.1 device)

I am using a SAMSUNG 780 500Gb for the drive, and that is where I am going to store my VM of Malcolm NSM.

There are a few ways to install Malcolm NSM

ISO install (Which I tried but failed, don't think I made it big enough)
Ubuntu Starting Image
From Source.

I picked #2.

I set up my VM with two NICs, 10Gb Memory, 4 cores, and 100Gb space (The first few times I tried this I ran out of space at 25Gb (hench the above comment about expanding my HD space on the machine)

With Ubuntu up and operational, it was time to start the installation.

sudo /scripts/install.py

And I got my first error (well it never really shows an error, but after digging, it looks like Docker is not being installed with the script). The error was trying to add a user to the docker group (which does not exist).

I tried to review the install.py but it pretty big file to review, so I decided to follow other guides in getting docker installed on Ubuntu Jammy InRelease.

sudo apt install apt-transport-https ca-certificates curl software-properties-common

 sudo mkdir -p /etc/apt/keyrings
 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

 echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update && sudo apt-get upgrade

sudo apt-get install docker-ce

Now I was able to run the install.py without any errors (A lot of responses were required though)

From there you have to do a reboot, and re-run install.py again.

After that, you run the command ./auth-setup

Then you run docker-compose pull (This is where I ran out of space the first time)

And finally ./start.

Maybe tomorrow I will write up initial impressions of Malcolm NSM

MalTrail - Day 2

2022-06-22T19:47:00.005-07:00

So back on working on MalTrail for the second day. I am able to access the web front end, but still no data is coming through.

First thing I tried was to edit the interfaces in the maltrail.conf from any to ens34 (The current sniffing interface on Security Onion). It might be better to use Bond0 as in 2.3 they use a bonded interface.

Restarting the docker (Docker restart maltrail). No new errors in the /var/log/maltrail/error.log, but also no traffic showing in the application.

Next step: docker exec -it maltrail bash to get into the Docker container. I am going to kill both python programs and try and restart them.

pkill server.py
pkill sensor.py
python server.py

Well starting server.py crashes the python program as it cannot find ens34. So now to the next question how do I bond a passive interface to a Docker container?

I decided to look at how the Suricata Docker container is being used.

docker inspect so-suricata

Looks like it's an environment variable setting it to the bond0.

Reading up on setting up a Docker container per-interface looks like you have to have an ip address associated with the interface (could be wrong).

Next, I read up on --net=host, but that might also cause other issues as the ports are dropped possibly between the Docker container and the host? (That's how I read it)

I do know that Security Onion uses a Docker network, but like before trying to keep things on the same server but separate.

MalTrail - SecurityOnion@Home (Try)

2022-06-21T23:52:00.004-07:00

I was reading up on MalTrail, and wanted to get that up and operational on my SecurityOnion VM (I know it probably can't handle it, but there is no fun in that). At the moment I am not sure how cpu/ram intensive MalTrail is, but I did see that it has been dockerized.

One thing of note, is I am going into this knowing that it will be unsupported by the SecurityOnion team, which is understandable.

So with that, I am playing on my home setup of SecurityOnion, which I often test things with.

Below is a screenshot of the Docker instructions.

Seems easy enough, I made a slight change to the export location. and skipped the apt parts :), since I am using CentOS, and would like to limit the application installed.

The for line I also skipped as the the sniffing port is already set to promisc

I could have added the Docker container into SecurityOnion elastic-net, but that could cause other issues down the road.

The install went pretty easy, no errors that I could see, but I could not access the site.

http://localhost:8338

My thoughts were that I needed to open the firewall for this. Reading up on SecurityOnion looks like I needed to create a Portgroup, and then add it to the INPUT for Standalone?

*This is a standalone install of SecurityOnion

so-firewall addportgroup maltrail
so-firewall addport 8338
salt system_standalone state.apply firewall

Tried again to access http://localhost:8338 (No Success)

Next I tried netstat -tulnp | grep LISTEN

Which looks like this:

This is the file I edited: /opt/so/saltstack/local/salt/firewall/assigned_hostgroups.local.map.yaml.

Reading more into this, I think I needed to add the ports to the Analyst Role, but I could be getting confused, cause I also see that its referencing the file: /opt/so/saltstack/local/pillar/minions/<HOSTNAME>_<ROLE>.sls

------------------

After this whole write-up, I figured it out. I used the documentation from SecurityOnion and this section.

Basically I added the maltrail port group to the Analyst role for the standalone machine through the standalone.sls file.

---Now to figure out why its not actually reporting anything.

SecurityOnion@Home - Alerts

2022-06-20T13:00:00.002-07:00

It's been a bit of time with my SecurityOnion server monitoring my network (out-of-band) and wanted to do an update on the type of alerts that I am seeing, and at least try to identify what some of them might be.

Below is a screen shot of some of the alerts. I also noticed that I am having issues Pivoting to PCAP inside SecurityOnion right now, not sure if it is something I possibly did, or just something with the .130 update (I have not done PCAP since the upgrade)

1. GPL WEB_SERVER: This looks like my Lenovo Smart Hub is trying to talk to something else on my network (Which is currently not on since I use DHCP, might switch them all to static for better tracking). But it looks like to trying to talk to these other devices about youtube.

2. I pay for Spotify for the family, so not really going to delve to deep on this one.

3. The next 2x are discord which I know is in use also

4. .cloud - Family is mostly Apple, and a quick look they are all the apple devices.

5. Microsoft Update - Validated

6. ET-DNS as Non-Compliant DNS traffic - Well this is my new cell phone, unsure what is doing.

7. DNS Query TOR Hidden (.onion) - Also my cell phone.

***Cell phone is Samsung 22 Ultra (Unlocked) on Tmobile. I will have to do some more research on these two to see what the phone is doing.

SecurityOnion@Home - Custom Alert Queries

2022-06-09T00:04:00.000-07:00

After the update to 2.3.130, I saw a discussion post about sorting and not staying on Alerts, which lead me down a rabbit hole about alerts. I noticed that there were a few prebuilt alerts, but how do I add new ones?

After a few minutes, I found a discussion post that referenced the alerts.queries.json file. Did some searching, and like all the other files that are customizable for the soc they are under:

/opt/so/saltstack/default/salt/soc/files/soc/

The three files currently are: alerts.queries.json, dashboard.queries.json, and hunt.queries.json.

An example was I added geodata to the alerts.queries.json

{ "name": "Group By Source IP/Port/Geo, Destination IP/Port/Geo, Name", "query": "* | groupby source.ip source.port source.geo.region_iso_code destination.ip destination.port destination.geo.region_iso_code rule.name" },

So I now have a new alert that can do checks on (This was just proof that it was available because at the time there was no reference in the documentation.

I took the next step and forked the Securityonion docs, and made a change to add the Custom Queries, which the SecurityOnion team merged with the official 2.3 documentation. After reviewing the update I saw that they added the dashboard/hunt queries reference (At the time I did not know those were also available)

SecurityOnion@Home - Mass Export PCAP?

2022-06-01T21:15:00.003-07:00

Another day, another task to try. I have a requirement to store PCAP offline, and n 16.04 there were folders for PCAP /nsm/*/dailylogs which I could rsync over to another storage device. In 2.3 there is a folder path for PCAP /NSM/PCAP but the files are not truly PCAP data files.

The files are created from Stenographer and can be converted back to PCAP (that's my understanding at least). That's how the SOC interfaces pull the PCAP based off of Alerts. But what if you wanted to copy all PCAP data.

Going through the SecurityOnion documentation and the discussion forums, I found mention of a script created by the SecurityOnion team called so-pcap-export. From the documentation:

I tested it with sudo so-pcap-export 'after 30m ago' output (leave off the .pcap as it is added in the script.

This created the file inside of the /nsm/pcapout folder. So this works for exporting, but it looks to only create one file at the moment. It would be nice to have the ability to create PCAP on set intervals, from the files created with Stenographer.

More searching led me to this script on the discussion forums for SecurityOnion.

https://github.com/Security-Onion-Solutions/securityonion/discussions/4038

Its a python script created by a Rob Hackworth, that I am still trying to get to work at the moment but it looks like it uses the same Stenoquery, just adds the ability to break up the PCAP by intervals. So far when I have tried it and it does create multiple PCAPs based off intervals. The only issue is that this version looks to only do by dates, and does not do by saying 24h, so I could copy days worth of PCAP.

One item of note from the script:

The time interval is critical as Steno will only let you export ~76G before it fails

Now I am not sure where that information comes from, maybe the users testing. The next set of testing would be to see how CPU intensive the converting of the files to PCAP is on the system, and then the addition of a possible rsync to this.

Currently trying to think of better ways, different ways to accomplish the task. I was thinking maybe using an NFS share and having the script copy the files directly to the shared folder (This might be heavy intensive, especially in the current build-out of our SecurityOnion.

I wonder if it's possible also someone set a marker on the last set of steno files that you converted back to PCAP so you are not recreating the same PCAP files.

SecurityOnion@Home - Adding Warning Banner

2022-05-30T21:28:00.000-07:00

I did a little digging on the SecurityOnion discussion forms and found out they have added a way to include a warning banner on the Login Page of Security Onion 2.3. It looks like it was an undocumented add with 2.3.30 (but was not stable as the unique path would get overwritten on updates.

In 2.3.50 they have added banner.md and it can be updated the same as the Motd.md (Part of the SOC Customization). The file itself is a Markdown file, and here is the Markdown Guide.

To customize the Login Banner content, copy banner.md as follows and then edit /opt/so/saltstack/local/salt/soc/files/soc/banner.md

sudo cp /opt/so/saltstack/default/salt/soc/files/soc/banner.md /opt/so/saltstack/local/salt/soc/files/soc/

SecurityOnion@Home - Suricata Include

2022-05-24T00:02:00.003-07:00

At work we started to use the include statement in our Suricata.yaml file while we were using SecurityOnion 16.04 and with the switch to SecurityOnion 2.3, I wanted to see how easy this would be to implement.

First things first, we go from having a Suricata.yaml for each interface to having one bonded interface, so that makes updating simple. With the bonded interface there is no more /etc/nsm/interface/suricata.yaml. I did find a suricata.yaml file under /opt/so/conf/suricata/ but it's created and managed with Salt, which means I probably cannot/should not edit that file directly.

I am running a standalone version of SecurityOnion at my house so the steps might be a little different in a distributed environment.

In the salt configuration first place I found a reference to the suricata yaml file was at:

/opt/so/saltstack/default/salt/suricata/default.yaml which build out the suricata.yaml to /opt/so/conf/suricata.

I decided to put the test_var.yaml @ /opt/so/conf/suricata/

Lets see what happens when I add the include file directly to the default.yaml file.

After the update you run: salt minionname_standalone state.highstate
Run: so-status

3. Run: so-suricata-start

4. Run: docker logs so-suricata

It's looking for the file in the /etc/suricata/ folder (Which does not exist on-base OS, but it does exist on the Docker container for Suricata.

We can validate this with docker inspect so-suricata (Looking for Binds)

As you can see that there is a /etc/suricata (The right side of the bind, which is the Docker container). Looks like for the most part there are files that are bound between the Docker image and the core OS of SecurityOnion.

I thought about putting the include file on the docker file @ /etc/suricata, but this does not sound like a good option for two reasons:

Upgrades would erase this
Probably more important, unsure if the included file inside the docker would work due to connecting the core OS.

From the last image it looks like I could potentially put the file in two directory locations:

/nsm/suricata ---> This is where the eve.json files are, so probably not the best location
/opt/so/conf/suricata/rules --> which maps to /etc/suricata/rules on the docker container

Back to salt to make some updates to the sls files. I am editing the /opt/so/saltstack/local/pillars/minions/name_standalone.sls

Note: Remember spacing (2) on yaml files

Edit the sls at the end:

suricata:
config:

include:/etc/suricata/rules/test_var.yaml

Next I copy the test_var.yaml -> /opt/so/conf/suricata/rules
Then I run: salt minionname_standalone state.highstate
Check Docker containers with so-status (All good)
And validate with docker logs so-suricata

Now I have an include file as part of SecurityOnion, and my local teams can update that file which could cause Suricata to fail, but it's better than them trying to update the sls files under minions to add/remove variables in my opinion (which could break more functionality besides Suricata)

SecurityOnion@Home: Sostat

2022-05-17T23:42:00.004-07:00

If you haven't noticed, my attention span jumps around a bit, it's currently focused on SecurityOnion 2.3, but I fall down a lot of rabbit holes, today it looks like sostat is what I was curious about. With that, sostat is one of the scripts that I currently miss from moving from 16.04 is sostat. It was widely used by the remote teams for troubleshooting. With a copy of sostat and maybe a few other choice logs it was easy to diagnose what was wrong with a remote system.

I know that there is a task in the backlogs to re-add sostat which would be great. Till then I had some time and started to try and re-create the magic. (It's not very good), but it gets the job done. Still struggling with trying to get Stenographer stats via the log file (Need some learning on cat/awk).

Most of it was just taking the 16.04 sostat and replaying it on my 2.3, locating file locations, changing a few things.

A few things I noticed so far based on 16.04 sostat: (Most of the ones here I asked on the Github Discussion to validate in the last day or so)

No sensortab file
No securityonion.conf
No interfaces folder breakouts (Per PCAP/Suricata)
I see mysql, but there is no mysql command (Do Uncategorized events exist?)

Easy ones to move over were (May not be 100% the same, or correct)

so-status
Link Statistics
Disk Space
CPU Usage
Version Info - Changed to: cat /etc/soversion
Zeek Packet - Changed to so-zeek-status (Might not be 100% the same)
Suricata Packet Drops - Changed LATEST_STATS to grep /opt/so/log/suricata.stats.log (Seems to work)

Ones that I could not figure out:

Stenographer - Tried to compare to the Netsniff (need to remind myself about sed/awk I think)
Network Sockets - No Lsof on SecurityOnion 2.3
Top 20 Alerts/Uncategorized
Log Archive - Doesn't look like it breaks out daily logs.

A preview of the tail end of it for now

SecurityOnion@Home: Splunk Forwarder

2022-05-17T08:18:00.000-07:00

So SecurityOnion 2.3 does come with Kibana, but I also like Splunk and want to get more experience with Splunk. My plan is to add a Splunk Forwarder to SecurityOnion. I have two paths I can take to accomplish this.

1. Install it on the machine as a normal application

2. Install a Docker Container of the Splunk Forwarder.

I have chosen two, cause one is too easy to complete. Plus this could give me more experience with Docker.

Step 1: docker pull splunk/universalforwarder:latest

Step 2: docker run --name uf --hostname uf -p 9997:9997 -e "SPLUNK_PASSWORD=<PASSWORD>" -e "SPLUNK_START_ARGS=--accept license" -e "SPLUNK_FORWARD_SERVER=XX.XX.XX.XX:9997" -d splunk/universalforwarder:latest

The docker container started, and is still operational, but don't see anything on my main Splunk Server. Also since I have the free version of Splunk, I do not have forwarder management, so can't see in Splunk if it's talking to each other.

I did try docker logs uf which looks like a bunch of ansible scripts, not the splunkd.log which I was looking for.

Next, I went and typed: docker exec -it -u splunk uf /bin/bash to access the container. I viewed the splunkd.log there but did find anything just yet of use.

I need to figure out how to add a folder to watch the docker container (Zeek/Suricata) for now. Might try and add the docker container to the SecurityOnion group of containers (If it benefits anything that is).

SecurityOnion@Home: Alerts Day 1

2022-05-09T11:14:00.007-07:00

After a few reloads I have SecurityOnion 2.3 on my home network. I have set up a TAP between my Netgear Mesh MR60 and the TP-LINK SG108 Switch. 90% of my traffic is WiFi, with only a few dedicated hard-wired machines.

Hardwired machines: My desktop, My Work Computer, Phillips Hue, and backhaul for one of the MS60 (Netgear Mesh).

Wifi: Everything else (TVs, Garage Door, Roku's, Apple TV, Phones (a lot), Tablets, 4x Laptops)

Currently, I am wondering if I should move the TAP from between the LAN/Switch, to the WAN/Modem side. From there I could possibly create a spanning port on the switch (Which is getting replaced soon with Zyxel GS1100 16) for the rest of the traffic.

Below is my current alerts (I cut off the IPs).

The first alert I noticed (not in the image) was:

ET POLICY Incoming Basic Auth Base64 HTTP Password detected unencrypted. (IP was my router).

* This was because my web access to the router was set at HTTP only, not HTTPS.

The next batch of alerts are below:

1. ET POLICY External Lookup Domain (myip.opendns) - I use OpenDns for my kids laptops for a some blocking.

2. ET POLICY HTTP Traffic on port 443 (This looks like it going to Amazon) - Identified.

3. ET TOR Know TOR relay/Router (Not Exit)/Exit Node Group 98 - Plex Server using the Remote Access Port identified in Settings

* Read that this could be NTP related, need to identify what it does

4. ET SCAN MS Terminal Server Traffic on Non-Standard Port - Plex Server using the Remote Access Port identified in Settings

5. ET TOR Known TOR exit Node Traffic group 76 - Plex Server using the Remote Access Port identified in Settings

I think I was expecting more alerts on my network, but it could be just the TAP placement.

Home Setup - Initial

2022-05-08T10:17:00.002-07:00

I have been using all these sites (TryHackMe, BlueLabsOnline, HacktheBox) to try and learn but I think the best way to get some knowledge is to monitor my own internet and see what kind of alerts I am getting and research them. The other day I purchased a new switch (16 ports) up from the 8 port switch that I currently use. Really I only needed like 10 ports but it was hard to find a 10 or 12-port switch. The Zyxel GS-1100 16 switch I purchased was used (40$ vice MSRP of about 100$).

I figured I could use it to create a spanning port for now, or try and acquire a cheap TAP (whereas I would not need the extra ports). In the TAP scenario, I figure the best place to TAP would be between the router and the modem. For spanning, I would span port 1 (the one coming from the router).

I already have an extra NIC in my main computer, and planning on building a new SecurityOnion 2.3 VM to start with. I thought of SecurityOnion 16.04, but believe it's better to go newer. I might also build a few other VMs. Currently thinking also of building a Splunk VM, but curious to think of what other VMs for monitoring traffic might be best.

I have seen other systems like OpenNSM, and DynamiteNSM, or I could try and roll something myself. That could be interesting, not sure I know what I would be doing or what I would want in the system.

SecurityOnion 16.04 - Squild

2022-04-27T23:44:00.002-07:00

Interesting issue arose today while trying to validate an installation of SecurityOnion 16.04. I was not seeing any traffic except SURICATA rules (even though ET PRO is installed). I decided to make my own rule in local.rules.

Get ready for some fun here, cause about 6Gb of ingest and I decided to write this without thinking:

alert any any -> any any (msg:"TEST TEST"; flow:established; classtype:misc_attack; sid:"999999"; rev:1;)

Then I ran sudo rule-update

Not starting that any any will just about kill the system as it alerts everything, but I also think the SID number was too low to use, which might have caused other issues.

Five minutes into it and I am seeing in Squert 1K alert, 10 minutes I am at 40K alerts, but interesting enough some ET PRO rules also are now firing. I removed the local.rules alert, reran sudo rule-update (no more TEST rules).

TEST Alert is now showing 150K in Squert.

Here are the commands I typed (From Suricata gone wild)

sudo so-sguild-stop
sudo mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db
SELECT COUNT(*) AS cnt, signature, signature_id FROM event WHERE status=0 GROUP BY signature ORDER BY cnt DESC LIMIT 20; (Top 20)

Interesting find here now under uncategorized events :

TEST Rule 150K 999999

Snort Alert [1:999999:1] 999999

So I have two alerts now for the same issues (This is because mid-stream I removed the TEST rule from my local.rules which deleted it out of my download.rules

UPDATE event SET status=1, WHERE event.status=0 and event.signature LIKE 'TEST%';
exit
sudo so-squid-start

This cleaned up the TEST alert, but I still had the Snort Alert. I wanted to wait for full processing then delete the Snort Alert to see if it came back

About an hour later I validated that traffic was done processing and that Snort Alert was still there.

sudo so-sguild-stop
sudo mysql --defaults-file=/etc/mysql/debian.cnf -Dsecurityonion_db
UPDATE event SET status=1, WHERE event.status=0 and event.signature_id='999999';
exit
sudo so-squid-start

My system was now clean from the ghost SID.

SecurityOnion 16.04 -- Strelka Install

2022-04-26T21:52:00.001-07:00

First and foremost, I know SecurityOnion 16.04 for EOL/EOS. We are trying to move away from it, it takes time. So I am trying to add new capabilities to the system. One is Strelka, which I did find a GitHub from Wes (SecurityOnion). Figured this would be an almost straightforward install (Never is).

Link to Github

Also found the old 16.04 documentation. Strelka was not a supported add-on to SecurityOnion (No Support), doesn't matter, stated earlier SecurityOnion 16.04 went EOL/EOS over a year ago now.

Next, I found an old ISO of Security Onion 16.04.6.2 (A few numbers off from the last official version). I am going to run setup for Stand-Alone(Basic setup Zeek/Suricata/Elastic). And then run a sudo soup before I start and try to make any changes to the system.

The previous VM I tried to run the Strelka install failed with Kibana issues, and then Docker issues so was unsure how this was going forward. Granted that was 16.04.2.

I ran the following from the terminal

wget https://raw.githubusercontent.com/weslambert/securityonion-strelka/master/install_strelka && sudo chmod +x install_strelka && sudo ./install_strelka

20 minutes later with a lot of RED, the install completed successfully per the output on the screen. The next step was to pump some data through it. With it being a new VM I didn't have it set up for traffic, so decided to just use sudo so-replay which pumped a lot of sample PCAP through the system.

Now the moment of truth, I opened up Kibana and looked for Strelka data (There is now a link in the left hand navigation window).

Clicked the link and it showed the next two images

So it seems to be sort of working, none of the other fields really filled in on this dashboard (could be the PCAP data was not good, or the actual install was not 100% installed). I guess I will have to go through the error logs and see if there is anything that makes sense.

FlipperZero: Repair Process Success

2022-04-24T16:36:00.001-07:00

It's been a day now that I have tried to get my Flipperzero up and running after trying to update the firmware. It has been stuck in DFU, unable to repair. I have been stuck at the below screen now for 2 days, and unfortunately, the forum post was not getting anywhere.

So on my primary computer, I reloaded the program twice, even tried switching to /dev branch of the qflipper program, same errors each time.

[RCY] Set Recovery boot mode @Or1sha START
[RCY] Set Recovery boot mode @Or1sha ERROR: Failed to set Recovery mode: operation timeout
[DEV] Full Repair @Or1sha ERROR: Failed to set Recovery mode: operation timeout

Next, I tried switching to multiple USB ports on my machine and verified in the device manager that the right drivers were installed for DFU in FS.

STM (Reading the forums though it references WinUSB, at the bottom, it says they are switching to the STM), so it could get confusing.

As a last resort, I pulled out another computer (Windows 10 also). Installed qflipper, and then I got a bunch of C++ redistributed errors.

vcruntime140_1.dll was not found

First I tried to run Windows Update, which took a few hours, but the computer has not been updated in a while. Still didn't work, so then I downloaded the new C++ files, rebooted, and put the flipper back in DFU.

Once again the flipper was recognized as needing repair. I clicked REPAIR, and it worked, it took about 3 minutes to run through all the processes but I am up and operational on my FlipperZero now.

Curious why my main machine did not like the USB, maybe there was a conflict that I was just not aware of, or something causing USB to disconnect after a few minutes.

FlipperZero: The Unboxing

2022-04-23T20:37:00.005-07:00

I just received the FlipperOne in the mail today (I pre-ordered it last year I believe for my birthday).

And the inside of the box (Flipper looks snug, and secure in the box it came in)

I turned it on just to mess with it for a few minutes before I decided I should update the firmware (I believe it came with .43 and it's now at .55)

So off I go and plug it into my computer, dowload the update program qflipper (for the flipperone) and start the update process. And it fails... Also now it won't turn on and does not show any indication of charging... Bricked in 2 minutes, not fun.

Reading up on the forums and I could try and get it into DFU mode, also read that it should have the microSD card in (I thought that was only if you were adding something like Kali, not a requirement for updates)

Got it into DFU (Don't worry I will make another post about getting it operational). And reran qflipper, comes up as needing repair. So at least not totally bricked. Clicked REPAIR, and its fails agian.
I grabbed the logs, and left a forum post, as I did not see any of the same direct errors that mine was reporting. Now I am waiting for support from the community.

--LOG--

[RCY] Set Recovery boot mode @Or1sha START

[RCY] Set Recovery boot mode @Or1sha ERROR: Failed to set Recovery mode: operation timeout [DEV] Full Repair @Or1sha ERROR: Failed to set Recovery mode: operation timeout

Blue Teams Labs Online: 6 Challenges Done

2022-04-17T12:24:00.006-07:00

Been using Blue Teams Labs Online for about a week now, and last night knocked out two more challenges. One the challenges had me stumped for a few days, I was way over thinking the question leading to multiple rabbit holes. Finally after multiple trys, I finally figured out what exactly it was asking and was able to provide the last answer for the challenge. So far each challenge (CTF, Digital Forensics, Incident Response) have been on the EASY path. I of course did not find them to easy, well for the most part say 80% of the challenge questions each were easy, they had the last one/two that have been stumping me. I need to remember that EASY does not mean to go find the most difficult way to solve the problem. The most obvious answer is probably the correct answer. Either way I have learned alot in just the six challenges and think next I will try and tackle an Investigation.

Blue Team Labs Online: Another Cyber Security Playground

2022-04-12T00:14:00.001-07:00

I found another site today called Blue Team Labs Online, looks to be a bunch of Defender type scenerios in there between investigations and challenges. I have just started to look over the challenges and have finished 1 out of many. I am stuck on two of them currently. Need to get my OSINT skills up and learn more about using Google for searching, and another about zip files which for the life of me I cant figure out at the moment. I dont want to post to much information about the challenges so I am being vague right now, but I am glad that I have found all these sites to learn from (BlueTeam Labs Online, Hack The Box, TryHackMe). I am sure there are more out there that are of use, and if anyone wants to comment for me to have a look at that would be great.

Udemy: Snort Intrusion Detection

2022-04-11T15:56:00.002-07:00

The other day I started a Udemy course: Snort Intrustion Detection, Rule Writing, and PCAP Analysis. In the lesson they use VirtualBox, I chose to use VMware, as that is what I have installed currently on my machine. Second was that a few of the sections were about setting up SecurityOnion and Kali (Both of which I already have active VMs for). That enabled me to shave off some of the time running through the training as they were setup as vanilla load, the only exception is that I have been using Suricata vice Snort, but for the most part I did not have an issue. One of the first rules was about SPAM, which we created a basic rule, and then added offset, and depth which he was able to explain. Suricata did not like the depth talked about in the video, stating it was shorter then the content (+1) to that number, and seemed to work with no ill effect. I need to read up more about offset/depth with reguards to Suricata, and see if it was just something I was doing wrong, or if there is a true differance between Snort and Suricata. Following that we did some other rules, and validated rules against VMs with known vulernabilities. I think this course helped a bit in understanding how the rulesets work, and will help with my current job.

version: "3"
	services:
	openvas:
	ports:
	- "8080:9392"
	environment:
	- "PASSWORD=admin"
	- "USERNAME=admin"
	- "RELAYHOST=172.17.0.1"
	- "SMTPPORT=25"
	- "REDISDBS=512" # number of Redis DBs to use
	- "QUIET=false" # dump feed sync noise to /dev/null
	- "NEWDB=false" # only use this for creating a blank DB
	- "SKIPSYNC=true" # Skips the feed sync on startup.
	- "RESTORE=false" # This probably not be used from compose... see docs.
	- "DEBUG=false" # This will cause the container to stop and not actually start gvmd
	- "HTTPS=false" # wether to use HTTPS or not
	volumes:
	- "openvas:/data"
	container_name: openvas
	image: immauss/openvas:$TAG
	volumes:
	openvas: