The post outlines the process and results of finetuning models to perform structured data extraction from press releases. The author reports that their finetuned models outperform OpenAI's GPT-4 models in terms of accuracy on several metrics. The process, however, involved considerable complexity and performance trade-offs. The
Table of contents
Adding predictions from our finetuned modelsJSON Validity TestStart Date AccuracyProvince AccuracyTarget Group AccuracyEvent Type AccuracyAccuracy for min_killedAccuracy for min_capturedAccuracy for killqAccuracy for captureqAccuracy for killcaptureraidAccuracy for airstrikeAccuracy for noshotsfiredAccuracy for min_leaders_killedAccuracy for min_leaders_capturedFinal aggregate scores for the modelsFinetuning works a charm, but…Sort: