With $1.76 million in pre-seed funding, SigScalr Inc., the creator of an open-source observability application meant to handle massive amounts of data, is making its debut today.
The Siglens platform, developed by a former observability engineer at Salesforce Inc., is a columnar analytical database that is said to process queries more than 1,000 times faster than Elasticsearch and more than 50 times quicker than Apache Clickhouse. Both are frequently applied to log analysis.
The key lies in a technique known as “micro-indexing,” in which an index is appended to every column, as described by creator and CEO Kunal Nawale. Immersion-proof compression modifies dynamically as new data comes in.
“In traditional databases, when you search for a value in a column, you search the column and put an index in front of it,” he said. “That index gets really large over time. Our index uses compression on the column, and the micro-index is 1/100th the size of a conventional index.” Columnar files are divided into smaller portions as a result, allowing for faster searching. During a search, performance engineers only need to uncompress 2% of the data.
The trade-off is that loglines cannot be altered in the past because the technology is append-only. That isn’t an issue in an observability environment, according to Nawale, though, because “you never edit the log line.”
Incredibly Scalable
Apart from its stated performance advantages, SigLens is supposed to be extremely scalable, enabling thousands of simultaneous searches on gigabytes of data with response times of less than a second. It can be used as software-as-a-service or on-premises to avoid cloud data egress fees. Each and every major observability tool is compatible with its query language.
“As data volumes grow above 1 terabyte a day, you can get to paying $15,000 a month on egress fees,” Nawale said. “We let you run inside your network so you don’t incur those costs.” He expects most users to opt for the on-premises version.
According to Nawale, he used a well-known big data platform at Salesforce for gathering, evaluating, and exploring machine-generated data when he had the concept for SigLens. “The vendor had no incentive to make the software more efficient,” he said. “We were spending several tens of millions of dollars on hardware.”
Older, all-purpose analytical databases are used by the majority of observability platforms, he stated. “We are solving this problem specifically for observability use cases.”
Only a few weeks ago, Siglens was made available as open source. The majority of SigScalr’s revenue is anticipated to come from the product’s hosted version as well as the enterprise edition’s additional security capabilities.
The round was led by Scribble Ventures Management LLC, with participation from Forward Slash Capital and WestWave Capital LLC.