# StormLib++ - An Intro

[`StormLib++`](https://github.com/gormaniac/stormlibpp) is a monorepo of sorts containing Python and Storm code that makes my life as a Storm developer and Synapse user easier. It's composed of two parts, a Python package [`stormlibpp`](https://pypi.org/project/stormlibpp/) and a [collection of Storm Packages](https://gormo.co/stormlibpp/pkgs/index.html) (Rapid Power-Ups). This blog post covers the evolution of the project and intends to highlight the main features it has to date. For more details on exactly what `StormLib++` can do - including API docs, a User Guide, and a Contribution Guide - checkout the [`StormLib++` documentation](https://gormo.co/stormlibpp/index.html).

`StormLib++` started as a way to portably collect some Python code used to make developing Storm Services easier. In particular, it made developing a Storm Package alongside its Storm Service better. Instead of embedding the Storm code directly in Python files, users could write a package Yaml file in its own folder and define Storm code in `.storm` files next to the Yaml definition, the way Rapid Power-Ups are typically written. A class was created (see [`stormlibpp.stormpkg.StormPkg`](https://gormo.co/stormlibpp/stormlibpp/stormlibpp.stormpkg.html#stormlibpp.stormpkg.StormPkg)) to load the traditionally developed Storm Package definition into a Python object - the Storm Service - at runtime. It even handled syntax checking of the Storm code as loaded, so that users wouldn't have to guess at where problems might be when their Storm Service is loaded by a Cortex. This feature was originally ported from the [`yarastorm` project](https://github.com/gormaniac/yarastorm).

`StormLib++` quickly grew into much more than a Storm Package loader. `v0.3.0` brought a Python definition of a Synapse Node ([`StormNode`](https://gormo.co/stormlibpp/stormlibpp/stormlibpp.node.html#stormlibpp.node.StormNode)) that was safe to read and modify within Python code before being "packed" and written back to a Cortex. `v0.4.0` came with an [`HttpCortex`](https://gormo.co/stormlibpp/stormlibpp/stormlibpp.httpcore.html#stormlibpp.httpcore.HttpCortex) object that implemented simple `synapse.cortex.Cortex` methods but over HTTP - something I needed in a network that wouldn't allow Telepath connections between users and a Cortex. `HttpCortex` has grown considerably since it was first released. At first, it only had `storm` and `callStorm` methods. Today it supports `addFeedData`, `exportStorm`, and `stormlist` as well.

`v0.5.x` added an HTTP-based Storm CLI ([`stormlibpp.hstorm`](https://gormo.co/stormlibpp/stormlibpp/stormlibpp.hstorm.html)). It actually relies on the same Python objects as `synapse.tools.storm`, but it uses `StormLib++`'s `HttpCortex` implementation. At first, this tool could only execute Storm statements. But more recent versions allowed the CLI to import and export `.node` files. It also supports the `onecmd` option to run Storm code directly from the command line and exit. Unfortunately, this tool cannot upload/download files from a Synapse Axon like the out-of-the-box Storm CLI. But hopefully one day, once the Axon HTTP API endpoints are fronted by the Cortex HTTP API, this can happen.

The feature I'm most proud of so far, a Storm script importer ([`stormlibpp.import`](https://gormo.co/stormlibpp/stormlibpp/stormlibpp.import.html)), came in `v0.6.x` and was perfected in subsequent versions. It works by recursively walking a directory, running each `.storm` file found, in alphabetical order, on a Cortex. It can read CSV, JSON, and text files of the same name as the Storm scripts as input for the given script. The idea was to improve upon the open source `synapse.tools.csvtool` features, and bring JSON and text file imports to the OSS Synapse community. This importer is feature rich and very powerful. A second blog post is in the works to cover it in depth - and even demonstrate how it can be used to manage more "static" Synapse data (user perms, tag definitions, taxonomies, etc.) as code.

`v0.10.x` was a big release for `StormLib++`. It added a new CLI tool, [`hfeed`](https://gormo.co/stormlibpp/stormlibpp/stormlibpp.hfeed.html). Which, if you can't already guess, is an HTTP-based version of `synapse.tools.feed`. It also made a change to `StormLib++`'s design, several Storm Packages (Rapid Power-Ups) were moved into the project's repo under the [`src/pkgs/` folder](https://github.com/gormaniac/stormlibpp/tree/main/src/pkgs). This was done for a few major reasons:
- Primarily, I didn't want to maintain multiple small repos for one off Synapse-related projects.
- I eventually wanted to add Github Actions to all of these projects, including `StormLib++`, to build and release code as new features are developed. Bringing these Storm Packages into `StormLib++` allows me to manage less GH Actions that would have ultimately been similar/the same.
- Moving all of these Storm Packages into `StormLib++` allowed me to namespace Storm Package commands and modules with an `slib` prefix. Something that's largely cosmetic, but I prefer it.

**What's next:**
I've got 2 big ideas brewing that I'd like to get added to `StormLib++`.  They're both currently partially implemented, I just need to find the time to work on them.

1) An improvement to the built-in Stix import features of Synapse ([PR #37](https://github.com/gormaniac/stormlibpp/pull/37)).
    - This is a custom import config, and a builder for the storm files that make up the config. There's also a mostly complete, but useless without the custom config being finished, Storm Package for this already present in recent versions of  `StormLib++`.
2) `stormgen` a Markdown -> Synapse Node converter ([PR #38](https://github.com/gormaniac/stormlibpp/pull/38)). This idea is pretty wild, and I've pontificated on it quite a bit in a [design proposal](https://github.com/gormaniac/stormlibpp/blob/stormgen/doc/stormgen_design.md) found in the `stormgen` branch. I encourage you to read this proposal doc if you're interested in learning more about it.
	- The main idea is that it is a tool that will be able to convert structured Markdown documentation into Storm node tuples that can be used with a Synapse Cortex's proxy methods to create or update nodes.
	- `stormgen` exists so that Synapse engineers/analysts don't have to do the double work of both documenting their data model and creating the nodes, which often contain elements of the same documentation. This allows users to document the expected usage, explain any rules or restrictions, warn others of gotchas, and even provide examples for any part of their Synapse data model without storing them in the model itself.
	- My hope is that it can also be used to long form document nodes in Markdown reports. Something that's very approachable for most analysts. The main use cases are reports that document threat actors, malware families, or observed activity and the IOCs seen along with it (like something observed by a security vendor or a report for a security event/incident at a company).

I don't think I'll consider `StormLib++` 1 dot 0 ready until some housekeeping (read better documentation, greater test coverage, and CI/CD) is complete. But in the meantime, existing features are stable and ready for use by all who are interested! I hope this project has been/can be useful to some. Especially those that are relying solely on the OSS Synapse features and those that are contributing to the growing 3rd-party OSS Synapse tool base.