Before changing anything, time every step on its own. The breakdown usually answers the question by itself: some steps need reasoning, and some are pattern matching wearing a costume.
Measure first, and the answer appears
01Classification 1.2sWhich document type is this? Pattern matching.
02Data extraction 2.2sPull names, dates, amounts. Named entity recognition.
03Clause analysis 4.2sGenuine reasoning. Keep the model.
04Risk assessment 2.2sGenuine reasoning. Keep the model.
05Summarisation 4.2sGenuine reasoning. Keep the model.
Run sequentially that is 14 seconds, and three of the five steps are using a language model for work a purpose-built model does faster and more accurately. Without measuring each step you would spend the week optimising the wrong ones.
Three changes, in order of payoff
- —Replace classification with a classifier — deciding between three document types is not reasoning.
- —Replace extraction with NER — pulling structured fields is a solved problem.
- —Parallelise what remains — risk and summary do not depend on each other, so they run together.
Done in that order, this approach can take a 14-second pipeline to roughly 4 seconds with no loss of output quality. Most of the gain comes from removing language models from steps that were never doing language work; parallelising is the smaller half.
