Specialized algorithms now routinely outperform human experts to greater than 90% accuracy and at greater persistent speeds for complex vision, language, and audio tests. Complex human cognition, however, hinges on chaining together many such specialized tasks into a larger workflow. For better machine training and human job emulation, this research explores the question of stacking multiple tasks as building blocks. We examine two complex multi-stage human tasks: 1) language translation from either text, imagery (optical character recognition), or scene context; and 2) cursor-on-target for object detection. The first task, a universal translator, carries science fiction roots, but this test requires multiple cognitive steps to recognize an arbitrary chunk of characters whether in text, email, images, or audio. Given a requirement to translate any of the 7,000 human languages, we explore whether domain expertise matters when designing a universal translator, particularly when compared to having many specialized workers performing a single task. The second cognitive task corresponds to a geo-intelligence job: given any location on earth, find its overhead satellite view and context, count or identify each type of object, then finally generate a representative caption summarizing the scene. The original contributions of this work find that the errors of single tasks propagate multiplicatively as more specialized tasks get chained together. We further find that in total, many well-defined but small goal-oriented models can outperform human experts when presented with a difficult job as end-to-end pipelines within the scope of modern machine learning. We finally propose a series of key human tasks for future work that share similar features of multiple (>3) application programming interfaces (API). When called sequentially, job tasking in this way can generate flexible and often unexpected performance metrics with rapid learning curves.
Published in
I/ITSEC 2021, USA
Publisher
I/ITSEC
Date of Conference
November 29–3, 2021