This talk was recorded at NDC Copenhagen in Copenhagen, Denmark. #ndccopenhagen #ndcconferences #developer #softwaredeveloper    

Attend the next NDC conference near you: 
https://ndcconferences.com
https://ndccopenhagen.com/

Subscribe to our YouTube channel and learn every day:   
/         @NDC 

Follow our Social Media!

https://www.facebook.com/ndcconferences
https://twitter.com/NDC_Conferences
https://www.instagram.com/ndc_conferences/

#dotnet #java #ai #ethics #functionalprogramming #javascript #softskills #genai #sql 

Like any reasonable human, I track my weekly cherry tomato consumption. But due to a tragic mix of bit flips, alternate realities, or maybe just my cat eating my spreadsheet, I lost the original data—leaving me with only screenshots of the plots.

It’s easy to generate plots from data. But what if you're stuck with the opposite problem: you have an image of a plot, and need to recover the underlying data?

In this lightning talk, I will show how we used a blend of computer vision, smart heuristics, and large language models (LLMs) to reverse-engineer data points from graph images. I will also share insights on how we tackled this problem for a real customer in the pharmaceutical industry, because the usefulness of this goes way beyond tomato tracking.

NDC Conferences

A system for automatically extracting data points from Kaplan-Meier graphs (survival curves) combines instance segmentation models, classical computer vision techniques, and LLMs. The approach uses fine-tuned vision transformers to isolate individual plot lines despite occlusions and crossings, morphological operations and skeletonization to identify event points at pixel level, and multimodal LLMs to extract metadata like axis units and legends. This enables mapping pixel coordinates to actual graph values, particularly useful for pharmaceutical research papers that publish graphs without underlying data.

Where Are My Tomatoes? Extracting Data from Graph Images with AI - Silas Eichsteller