Issue #4: LLMs can revitalize dying languages

It’s Thursday, and time for another issue of General Intelligence!

In this issue, we cover

• IBM trains language models to revitalize dying languages

• LLMs are more accurate than search engines when answering health questions

• Agents that can automate IT ops

• How to teach LLMs to say “No” and not comply with a user prompt

• 43 questionable research practices researchers use when publishing work

As always, the links to the research papers are provided

📣 IBM Research says LLMs can revitalize dying languages

IBM Research Brazil explore have been working to create AI technologies for indigenous peoples in Brazil, targeting small communities where indigenous languages are still in use but under threat. About 200 languages are spoken currently in Brazil by between one to two million people but the vast majority of these languages are in danger of disappear until the end of the century^.

The researchers created an Indigenous Language Model (ILM) as a replicable and scalable way to create spell-checkers, next-word predictors, and similar tools. arxiv

🩻 LLMs vs search engines: Which is better at answering health questions?

This study compares the effectiveness of search engines, large language models (LLMs), and retrieval-augmented generation (RAG) approaches in answering health-related questions. It finds that LLMs generally provide more accurate answers than traditional search engines, though they are sensitive to input prompts. The research also indicates that RAG methods are highly effective, and the quality of webpages found by search engines does not significantly decline with lower rankings. arxiv

🖥️ AI for IT Ops: automating complex tasks with agents

Researchers at Microsoft, Cal, and UIUC address the growing interest in AI for IT Operations (AIOps), which aims to automate complex tasks like fault localization and root cause analysis to minimize human intervention and customer impact. It highlights the lack of standardized frameworks for developing, evaluating, and improving AIOps agents as a major obstacle to achieving autonomous, self-healing clouds. The authors propose AIOpsLab, a prototype framework that uses chaos engineering to inject real-time faults and interfaces with agents to localize and resolve these issues, thereby providing a foundation for AIOps development. arxiv

🙅‍♂️ Teaching AI to Say 'No': New Research Tackles Noncompliance in Language Models

Researchers at the Allen Institute for AI, Microsoft, et al introduce a taxonomy for when language models should refuse user requests beyond just unsafe queries, covering categories like incomplete, unsupported, indeterminate, and humanizing requests. It develops a new evaluation suite of 1,000 prompts to test noncompliance and finds that models like GPT-4 still comply incorrectly in many cases. The authors propose training strategies, including low rank adapters, to improve noncompliance while maintaining model capabilities. arxiv

🔍 Fraud in ML research? 43 questionable research practices

This paper lists 43 questionable research practices (QRP) in Machine Learning, that can borderline on research fraud. As researchers are incentivized to publish state-of-the-art results, they may make it difficult or impossible to reproduce research results.

arxiv

🤯 Today I Learned

Every issue, we highlight new AI concepts and terminology to help educate our readers. This issue we learned about:

AIOps

AIOps, short for Artificial Intelligence for IT Operations, refers to the use of artificial intelligence and machine learning technologies to enhance and automate IT operations processes. Some benefits of AIOps include

• Faster incident response and resolution

• Proactive problem prevention

• Improved resource allocation

• Enhanced visibility across complex IT environments

Few Shot Learning

Few-shot learning is a machine learning approach that enables models to learn from a very small number of examples, typically 1-5 examples. It contrasts with traditional machine learning methods that often require large datasets for training. Few-shot learning aims to mimic human-like ability to quickly grasp new concepts from limited exposure. This approach is particularly valuable in scenarios where collecting large datasets is impractical, expensive, or time-consuming.