A new report from Google’s DeepMind outlines a new method of evaluating how AIs understand human language. DeepMind researchers, led by Elena Gribovskaya and Tomáš Kočiský, presented their findings in a paper that has interesting implications for AI research.
One kind of useful AI is a bot that can read and ‘understand’ human language in the form of written information. These types of bots help us find the information we need by scanning the vast amounts of information available online and selecting the most relevant and useful answers. We see this kind of work performed by web search algorithms and virtual assistants.
These kinds of AIs (referred to as ‘language models’) are typically evaluated for knowledge and language understanding via question answering (QA), i.e., they answer questions based on a piece of information, such as a Wikipedia or news article. However, knowledge does not remain static over time, rather, it grows and evolves with additions and revisions, so the knowledge of these bots becomes outdated.
DeepMind constructed a new large-scale dataset, called StreamingQA, to test how models could adapt to evolving knowledge. It used human-composed questions and gave the AI 14 years of time-stamped news articles to draw information from for its answers. The models were evaluated on a quarterly basis as they read new articles and updated their knowledge base.
The results showed that both parametric and semi-parametric models benefitted in accuracy from StreamingQA, being able to adapt to new information without full retraining. It was particularly useful for high-frequency topics that continue to be relevant over long periods of time. In a world of rapidly-updating, ever-expanding information where we increasingly rely on technology to sort signals from noise, StreamingQA and methods like it could help our virtual assistants and web searches be more effective.
To read the original news article, click here
Please rate this post
AI Optimization Disruptive Researcher – Chief Development Officer and CoFounder at AccelOne – Blockchain Certified Developer – Autonomous Cars Engineer – Industrial Engineer – McLaren Fan
My Tech journey started 38 years ago with a Yamaha CX5M Computer / Synth, and since then, I have had a nonstop career adding experiences in several industries and technologies. I am an AI researcher and Ph.D. student and the Chief Development Officer of AccelOne. I lead a team of passionate talent in software engineering services, which are delivered from LA to the US. I have a strong technical background with more than three decades of experience in complex project and team management in various areas, including product design and development in Entertainment, eCommerce, Retail, Logistics, Business Intelligence, and Financial Services. I was CTO of Axigma Technologies, managing mobile business and consumer development projects for brands, including The Marketing Store and C9W. I founded the computer training institute IEC, which provided training services in several different programming languages and design and animation tools. In 2005 I founded Routeck, a development software company devoted to special projects (such as open-source firmware programming), credit card reconciliation, and specific products for retail. I was Development Manager at Infinite Corporation, managing their iSeries and Web products, a former Senior Software Engineer at COTO, and a Senior Web Applications Engineer at HSBC.