What
I’m building streaksight, a GUI-based data analysis tool that runs locally on your machine.
The project is open source and available on GitHub: https://github.com/whtsht/streaksight
The goal is to minimize the time required to turn data into insights. streaksight lets you load, analyze, and visualize data in a single local application.
Note: streaksight is still a work in progress and hasn’t been released yet.
What you can do
Data ingestion
By simply selecting a file, streaksight automatically detects the schema and imports the data into its embedded database. Currently, CSV and JSON formats are supported. In the future, the following data sources are planned:
- Object storage such as AWS S3
- Databases such as MySQL …
What
I’m building streaksight, a GUI-based data analysis tool that runs locally on your machine.
The project is open source and available on GitHub: https://github.com/whtsht/streaksight
The goal is to minimize the time required to turn data into insights. streaksight lets you load, analyze, and visualize data in a single local application.
Note: streaksight is still a work in progress and hasn’t been released yet.
What you can do
Data ingestion
By simply selecting a file, streaksight automatically detects the schema and imports the data into its embedded database. Currently, CSV and JSON formats are supported. In the future, the following data sources are planned:
- Object storage such as AWS S3
- Databases such as MySQL
- SaaS platforms such as Google Analytics
Query building and visualization with nodes
Queries can be constructed by connecting nodes. Execution results are shown in real time, allowing you to iteratively explore data and gain insights interactively while refining your queries.
Who
streaksight is best suited for working with relatively small datasets.
- Indie developers and small teams running their own products
- People who want to analyze small to medium-sized log or event data locally
- Teams who want to validate the value of their data before building a full-scale data platform
Why
I first encountered DuckDB while investigating logs at work. Its execution speed was surprisingly fast and left a strong impression on me.
After that experience, I started following the DuckDB ecosystem more closely. In particular, I’m paying close attention to projects like MotherDuck and DuckLake.
At the same time, I was responsible for operating BigQuery including cost optimization efforts. Through this work, I became increasingly interested in the technologies behind building and operating data platforms. I explored various SaaS and OSS projects related to data infrastructure, and during that process, I noticed several recurring problems.
Problem 1: Setup complexity
Most data infrastructure OSS projects focus on a single responsibility and are designed to be composed together.
ETL: Airbyte, Apache NiFi, ...
Query engines: DuckDB, ClickHouse, ...
Visualization: Redash, Metabase, Apache Superset, ...
This separation improves extensibility and stability, but it also increases setup and maintenance costs. Selecting the right tools and wiring them together can add noticeable overhead, especially for small-scale data analysis.
Problem 2: Difficulty of building correct queries
With the rise of AI, writing SQL queries has become much easier. However, debugging those queries and verifying that they actually produce the intended results remains a difficult task.
A query is a sequence of set operations and joins. To quickly validate AI-generated queries, I felt there was a strong need for an interface that allows users to inspect intermediate results, such as subqueries and join outputs.
streaksight addresses these problems by enabling fast, iterative query experimentation and result inspection in a local environment, without combining multiple tools.
Technology stack
Design principle: run efficiently on a single machine
DuckDB An OLAP-focused database that delivers excellent performance on a single machine
Tauri Enables building cross-platform desktop applications Produces lightweight binaries and allows the frontend to be built with web technologies
Deno.js A JavaScript runtime that integrates well into Rust-based projects Deno is used to allow users to add custom JavaScript plugins without rebuilding the application.
Singer.io Used as a reference when designing the interface for ETL plugins
Roadmap
Because data sources vary widely, flexibility in connectors is a key focus. The next steps are to increase the number of supported connectors and to finalize a plugin interface. Ultimately, the goal is to allow users to write custom connectors in JavaScript, enabling direct connections to a wide variety of data sources.
In addition, the following features are being explored:
Sharing insights
- Dashboards
AI integration
- Model Context Protocol
Cloud integration
The scope and direction of these features will be refined based on real-world use cases.
Final words
If you find this project interesting, I would really appreciate it if you could give it a star on GitHub.
If you have any questions or feedback about the project, please feel free to start a discussion here: https://github.com/whtsht/streaksight/discussions