Show HN: Robust LLM Extractor for HTML/Markdown in TypeScript
(github.com/lightfeed)
Use LLMs to robustly extract structured data from HTML and markdown.
Use LLMs to robustly extract structured data from HTML and markdown.
Cell Mates: Extracting Useful Information from Tables for LLMs
(gojiberries.io)
We seem to have cracked the art of distilling information in words and images using large machine learning models. But our ability to exploit useful information in tabular data using large models is mostly missing.
We seem to have cracked the art of distilling information in words and images using large machine learning models. But our ability to exploit useful information in tabular data using large models is mostly missing.
Why extracting data from PDFs is still a nightmare
(arstechnica.com)
For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files.
For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files.
Show HN: OCR pipeline for ML training (tables, diagrams, math, multilingual)
(github.com/ses4255)
This OCR system is specifically designed to extract structured data from complex educational materials—such as exam papers—in a format optimized for machine learning (ML) training.
This OCR system is specifically designed to extract structured data from complex educational materials—such as exam papers—in a format optimized for machine learning (ML) training.
Why extracting data from PDFs is still a nightmare for data experts
(arstechnica.com)
For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files.
For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files.
Why extracting data from PDFs is still a nightmare for data experts
(arstechnica.com)
For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files.
For years, businesses, governments, and researchers have struggled with a persistent problem: How to extract usable data from Portable Document Format (PDF) files.
Show HN: A lightweight LLM proxy to get structured results from most LLMs
(l1m.io)
l1m is the easiest way to get structured data from unstructured text or images using LLMs. No prompt engineering, no chat history, just a simple API to extract structured json from text or images.
l1m is the easiest way to get structured data from unstructured text or images using LLMs. No prompt engineering, no chat history, just a simple API to extract structured json from text or images.
Run structured extraction on documents/images locally with Ollama and Pydantic
(github.com/vlm-run)
Welcome to VLM Run Hub, a comprehensive repository of pre-defined Pydantic schemas for extracting structured data from unstructured visual domains such as images, videos, and documents.
Welcome to VLM Run Hub, a comprehensive repository of pre-defined Pydantic schemas for extracting structured data from unstructured visual domains such as images, videos, and documents.
From PDFs to Insights: Structured Outputs from PDFs with Gemini 2.0
(philschmid.de)
This week Google DeepMind released Gemini 2.0, including Gemini 2.0 Flash (General Available), Gemini 2.0 Flash-Lite (New cost-efficient) and Gemini 2.0 Pro (Experimental). All models support up to at least 1 million input tokens with support for text, images and audio and function calling/structured outputs.
This week Google DeepMind released Gemini 2.0, including Gemini 2.0 Flash (General Available), Gemini 2.0 Flash-Lite (New cost-efficient) and Gemini 2.0 Pro (Experimental). All models support up to at least 1 million input tokens with support for text, images and audio and function calling/structured outputs.
PDF Hell: Why Is Extracting Data Still a Nightmare? (2024)
(unstract.com)
If you have tried to extract text from PDFs you would have come across a myriad of complications related to it. It is relatively easy to do a POC or experiment, but when it comes to handling PDFs from the real world on a consistent basis, it is a tremendously difficult problem to solve.
If you have tried to extract text from PDFs you would have come across a myriad of complications related to it. It is relatively easy to do a POC or experiment, but when it comes to handling PDFs from the real world on a consistent basis, it is a tremendously difficult problem to solve.
Fun building better table extraction models
(aryn.ai)
Aryn’s goal is to provide analytics over unstructured documents, typically PDFs. We believe that integral to that goal is the ability to break down these documents into a more structured form that we can use to extract more information than you might be able to by simply stripping out all of the text and embedding it with a language model.
Aryn’s goal is to provide analytics over unstructured documents, typically PDFs. We believe that integral to that goal is the ability to break down these documents into a more structured form that we can use to extract more information than you might be able to by simply stripping out all of the text and embedding it with a language model.
Nvidia-Ingest: Multi-modal data extraction
(github.com/NVIDIA)
NVIDIA-Ingest is a scalable, performance-oriented document content and metadata extraction microservice. Including support for parsing PDFs, Word and PowerPoint documents, it uses specialized NVIDIA NIM microservices to find, contextualize, and extract text, tables, charts and images for use in downstream generative applications.
NVIDIA-Ingest is a scalable, performance-oriented document content and metadata extraction microservice. Including support for parsing PDFs, Word and PowerPoint documents, it uses specialized NVIDIA NIM microservices to find, contextualize, and extract text, tables, charts and images for use in downstream generative applications.
Liberate tabular data from scanned documents
(blog.wzb.eu)
During the last months I often had to deal with the problem of extracting tabular data from scanned documents.
During the last months I often had to deal with the problem of extracting tabular data from scanned documents.
Gemini Flash 2.0 Experimental: A bit more accurate, but slower
(ycombinator.com)
I just updated my data extraction leaderboard with Gemini Flash 2.0 Experimental. It is quite a bit slower with large input token sizes right now, but a bit more accurate than Gemini 1.5 Flash which is at the top right now.
I just updated my data extraction leaderboard with Gemini Flash 2.0 Experimental. It is quite a bit slower with large input token sizes right now, but a bit more accurate than Gemini 1.5 Flash which is at the top right now.
Show HN: Get any domain's brand data via API
(brand.dev)
Get logos, primary colors, descriptions, classifications, and more from any domain with a single api call.
Get logos, primary colors, descriptions, classifications, and more from any domain with a single api call.
Scaling Document Data Extraction with LLMs and Vector Databases
(timescale.com)
Extracting structured data from unstructured documents is a powerful use case for large language models (LLMs). This sort of data extraction from complex documents has always remained a challenge. Done either completely manually or using current intelligent document processing (IDP) platforms that utilize previous-generation machine learning or natural language processing (NLP) techniques is very time-consuming and tedious.
Extracting structured data from unstructured documents is a powerful use case for large language models (LLMs). This sort of data extraction from complex documents has always remained a challenge. Done either completely manually or using current intelligent document processing (IDP) platforms that utilize previous-generation machine learning or natural language processing (NLP) techniques is very time-consuming and tedious.
Maxun: Open-Source No-Code Web Data Extraction Platform
(github.com/getmaxun)
Maxun lets you train a robot in 2 minutes and scrape the web on auto-pilot. Web data extraction doesn't get easier than this!
Maxun lets you train a robot in 2 minutes and scrape the web on auto-pilot. Web data extraction doesn't get easier than this!
Show HN: Tile.run – Extract structured data from any document via API
(tile.run)
tile.run is the simplest way to extract data from any document with best in class accuracy.
tile.run is the simplest way to extract data from any document with best in class accuracy.
Launch HN: Midship (YC S24) – Turn PDFs, docs, and images into usable data
(ycombinator.com)
Hey HN, we are Max, Kieran, and Aahel from Midship (https://midship.ai). Midship makes it easy to extract data from unstructured documents like pdfs and images.
Hey HN, we are Max, Kieran, and Aahel from Midship (https://midship.ai). Midship makes it easy to extract data from unstructured documents like pdfs and images.
Video scraping: extracting JSON from a 35s screen capture for 1/10th of a cent
(simonwillison.net)
The other day I found myself needing to add up some numeric values that were scattered across twelve different emails.
The other day I found myself needing to add up some numeric values that were scattered across twelve different emails.
Extracting financial disclosure and police reports with OpenAI Structured Output
(github.com)
Instantly share code, notes, and snippets.
Instantly share code, notes, and snippets.
Show HN: Pipet – CLI tool for scraping and extracting data online, with pipes
(github.com/bjesus)
Pipet is a command line based web scraper. It supports 3 modes of operation - HTML parsing, JSON parsing, and client-side JavaScript evaluation. It relies heavily on existing tools like curl, and it uses unix pipes for extending its built-in capabilities.
Pipet is a command line based web scraper. It supports 3 modes of operation - HTML parsing, JSON parsing, and client-side JavaScript evaluation. It relies heavily on existing tools like curl, and it uses unix pipes for extending its built-in capabilities.
Table Extraction Using LLMs
(nanonets.com)
Picture this - you’re drowning in a sea of PDFs, spreadsheets, and scanned documents, searching for that one piece of data trapped somewhere in a complex table.
Picture this - you’re drowning in a sea of PDFs, spreadsheets, and scanned documents, searching for that one piece of data trapped somewhere in a complex table.
Bitten by Unicode
(pyatl.dev)
One product of mine takes reports that come in as a table that’s been exported to PDF, which means text extraction. For dollar figures I find a prefixed dollar symbol and convert the number following it into a `float`. If there’s a hyphen in addition to the dollar symbol, it’s negative.
One product of mine takes reports that come in as a table that’s been exported to PDF, which means text extraction. For dollar figures I find a prefixed dollar symbol and convert the number following it into a `float`. If there’s a hyphen in addition to the dollar symbol, it’s negative.