Calling All Machine Learning Researchers

Analytics, machine learning, and artificial intelligence may not be new, but AI and machine learning are experiencing a rapid uptake among organizations. The interest by businesses and other enterprises has led one organization to embark on a bit of a reality check and an effort to document the state of AI and machine learning and projects so far.

"Many people are starting to ask what a world with intelligent computers will look like," Peter Eckersley and Gennie Gebhart of the Electronic Frontier Foundation write in a blog post. "But what is the ratio of hype to real progress? What kind of problems have been well solved by current machine learning techniques which ones are close to being solved, and which ones remain exceptionally hard?"

The EFF, an international non-profit digital rights advocacy and legal defense organization, is launching a new program called the EFF AI Progress Measurement experiment. This pilot project collects problems and metrics/datasets from the AI research literature and tracks progress on them, according to the EFF.

"At EFF, we're ultimately most interested in how this data can influence our understanding of the likely implications of AI," the EFF writes on the project page. "To begin with, we are focused on gathering it."

The initial project is here, and it documents many of the AI and machine learning projects to date.

The EFF is looking for help and contributors to this project. The organization is calling on machine learning researchers to provide feedback and contribute to the effort. So far EFF has drawn data from sources including blog posts, web sites, and review articles, and collected the information on Github.

Now EFF is asking machine learning researchers to weigh in on the following questions: Is this information useful to the machine learning community? And, what important problems, datasets, and results are missing from what's there so far?

Let us know in the comments what you think of the state of machine learning to date, and of EFF's effort to document it.