In the next ten years, superintelligence might emerge, in our opinion. These artificial intelligence (AI) systems would be extremely powerful and potentially very risky.
These days, we employ reinforcement learning from human feedback (RLHF) to align AI systems to guarantee their safety. However, there will be essentially new and distinct technical obstacles to coordinating future superhuman AI systems.
AI systems that are superhuman will be able to perform difficult and imaginative tasks that are beyond the comprehension of humans. It’s possible that human supervision-based alignment methods like RLHF are no longer adequate. This brings up the main problem: how can people control and put their trust in AI systems that are far smarter than them?
One of the most significant unresolved technical issues in the world is this one. However, we believe that it is solvable with a focused effort. There are tons of low-hanging fruit opportunities, fascinating new ideas, and promising techniques.
OpenAI Superalignment Fast Grants
|Male and Female
|February 18, 2024
Goals and Benefits of OpenAI Super-alignment Fast Grants
- $100K to $2M in grants are available for independent researchers, charitable organizations, and academic labs.
- A $150K OpenAI Superalignment Fellowship, consisting of a $75K stipend and $75K in computing and research money, is being sponsored for graduate students.
Criteria for OpenAI Super-alignment Fast Grants
We would especially like to fund the following research directions with these grants:
- Generalization from weak to strong: Compared to superhuman models, humans will make poor supervisors. How strong models generalize from weak supervision is something we can understand and influence.
- Interpretability: What is the best way to comprehend the internals of a model? Can we utilize this, for example, to create an artificial intelligence lying detector?
- Scalable oversight: How might AI help humans assess other AI systems’ performance on difficult tasks?
- Many other study directions exist, such as adversarial robustness, chain-of-thought faithfulness, honesty, evaluations and test-beds, and more.
Deadline for Application
February 18th, 2024
How to Apply
Interested and qualified? Go to OpenAI on airtable.com to apply
- They expect to be able to make grants to individuals and institutions in most other countries as well, barring legal restrictions.
- No prior experience working on alignment is required; they are actively looking to support researchers who are excited to work on alignment for the first time.
- The application process is simple, and they’ll get back to you within four weeks of applications closing.
For more details,visit OpenAI website