Show HN: Hamilton's UI – observability, lineage, and catalog for data pipelines

https://github.com/DAGWorks-Inc/hamilton/tree/main/ui

Skip to content

Navigation Menu

Sign in

Appearance settings

You can’t perform that action at this time.

{
"by": "elijahbenizzy",
"descendants": 10,
"id": 40235944,
"kids": [
40237169,
40244558,
40237525,
40244753,
40249414
],
"score": 40,
"text": "Hey HN – Stefan and Elijah here from DAGWorks (<a href=\"http:&#x2F;&#x2F;dagworks.io&#x2F;\">http:&#x2F;&#x2F;dagworks.io&#x2F;</a>, YC W23).<p>If you don’t remember us from our previous HN launch (<a href=\"https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35056903\">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35056903</a>), we’re the authors of Hamilton (<a href=\"https:&#x2F;&#x2F;github.com&#x2F;dagworks-inc&#x2F;hamilton\">https:&#x2F;&#x2F;github.com&#x2F;dagworks-inc&#x2F;hamilton</a>), an open-source library for building self-documenting, modular dataflows in python that works for data, ML, LLM pipelines, &amp; even web-workflows.<p>We’ve been developing this UI for a while and we’re excited to say we open-sourced it! It comes out of the box with the following capabilities, and only requires a single line code change to get:<p>1. Execution + metadata capture, e.g. automatic code profiling<p>2. Data&#x2F;artifact observability, e.g. summary statistics over dataframes, pydantic objects, etc...<p>3. Lineage &amp; provenance of data, e.g. quickly see what is upstream &amp; downstream of code&#x2F;data.<p>4. Asset&#x2F;transform catalog, e.g. search &amp; find if feature transforms&#x2F;metrics&#x2F;datasets&#x2F;models exist and where they’re used.<p>While the UI currently only self-populates for Hamilton dataflows, we’re looking to expand to other frameworks (we’d love your feedback!).<p>Check out the following video for an overview: <a href=\"https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=0VIVSeN7Ij8\" rel=\"nofollow\">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=0VIVSeN7Ij8</a>, as well as the documentation: <a href=\"https:&#x2F;&#x2F;hamilton.dagworks.io&#x2F;en&#x2F;latest&#x2F;concepts&#x2F;ui&#x2F;\">https:&#x2F;&#x2F;hamilton.dagworks.io&#x2F;en&#x2F;latest&#x2F;concepts&#x2F;ui&#x2F;</a>.<p>We’re looking for feedback&#x2F;adopters – feel free to reach out if you have any questions!",
"time": 1714656159,
"title": "Show HN: Hamilton's UI – observability, lineage, and catalog for data pipelines",
"type": "story",
"url": "https://github.com/DAGWorks-Inc/hamilton/tree/main/ui"
}
{
"author": "apache",
"date": null,
"description": "Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does. - apache/…",
"image": "https://opengraph.githubassets.com/c9d53249867b737c8c5f81e90d99c06c319d2deb4c8cf2acd59cf4ceccb2b4fb/apache/hamilton",
"logo": null,
"publisher": "GitHub",
"title": "hamilton/ui at main · apache/hamilton",
"url": "https://github.com/apache/hamilton/tree/main/ui"
}
{
"url": "https://github.com/apache/hamilton/tree/main/ui",
"title": "hamilton/ui at main · apache/hamilton",
"description": "Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does. - apache/...",
"links": [
"https://github.com/apache/hamilton/tree/main/ui",
"https://github.com/DAGWorks-Inc/hamilton/tree/main/ui"
],
"image": "https://opengraph.githubassets.com/c9d53249867b737c8c5f81e90d99c06c319d2deb4c8cf2acd59cf4ceccb2b4fb/apache/hamilton",
"content": "<div>\n <div>\n <p><a target=\"_blank\" href=\"https://github.com/apache/hamilton/tree/main/ui#start-of-content\">Skip to content</a>\n <span>\n <span></span>\n</span></p>\n <h2>Navigation Menu</h2>\n <div>\n <div>\n <a target=\"_blank\" href=\"https://github.com/\">\n </a>\n <div>\n <p><a target=\"_blank\" href=\"https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapache%2Fhamilton%2Ftree%2Fmain%2Fui\">\n Sign in\n </a></p><div>\n Appearance settings\n </div>\n </div>\n </div>\n <div>\n <div><ul><li><div><ul><li><div><p><span>AI CODE CREATION</span></p><ul><li><a target=\"_blank\" href=\"https://github.com/features/copilot\"><div><p><span>GitHub Copilot</span><span>Write better code with AI</span></p></div></a></li><li><a target=\"_blank\" href=\"https://github.com/features/spark\"><div><p><span>GitHub Spark</span><span>Build and deploy intelligent apps</span></p></div></a></li><li><a target=\"_blank\" href=\"https://github.com/features/models\"><div><p><span>GitHub Models</span><span>Manage and compare prompts</span></p></div></a></li><li><a target=\"_blank\" href=\"https://github.com/mcp\"><div><p><span>MCP Registry<sup>New</sup></span><span>Integrate external tools</span></p></div></a></li></ul></div></li><li></li><li></li><li></li></ul><p><a target=\"_blank\" href=\"https://github.com/features\"><span>View all features</span></a></p></div></li><li></li><li></li><li></li><li></li><li><a target=\"_blank\" href=\"https://github.com/pricing\"><span>Pricing</span></a></li></ul></div>\n <div>\n <div>\n <div>\n <p>\n </p><h2 id=\"feedback-dialog-title\">\n Provide feedback\n </h2>\n <p></p>\n </div>\n <div>\n <p>\n </p><h2 id=\"custom-scopes-dialog-title\">\n Saved searches\n </h2>\n <h2 id=\"custom-scopes-dialog-description\">Use saved searches to filter your results more quickly</h2>\n <p></p>\n </div>\n </div>\n <p><a target=\"_blank\" href=\"https://github.com/signup?ref_cta=Sign+up&amp;ref_loc=header+logged+out&amp;ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Ffiles%2Fdisambiguate&amp;source=header-repo&amp;source_repo=apache%2Fhamilton\">\n Sign up\n </a></p><div>\n Appearance settings\n </div>\n </div>\n </div>\n </div>\n </div>\n <div>\n <div>\n <div>\n <ul>\n <li>\n <a target=\"_blank\" href=\"https://github.com/login?return_to=%2Fapache%2Fhamilton\"> Notifications\n</a> You must be signed in to change notification settings\n </li>\n <li>\n <a target=\"_blank\" href=\"https://github.com/login?return_to=%2Fapache%2Fhamilton\"> Fork\n <span>172</span>\n</a>\n </li>\n <li>\n <p>\n <a target=\"_blank\" href=\"https://github.com/login?return_to=%2Fapache%2Fhamilton\"> <span>\n Star\n</span> <span>2.4k</span>\n</a></p>\n </li>\n</ul>\n </div>\n </div>\n </div>\n <div>\n <p>\n You can’t perform that action at this time.\n </p></div>\n <details>\n <summary></summary>\n </details>\n <p>\n </p>\n <p>\n </p>\n </div>",
"author": "",
"favicon": "https://github.githubassets.com/favicons/favicon.svg",
"source": "github.com",
"published": "",
"ttr": 14,
"type": "object"
}