Generative AI models
Foundation Models
- Large amount of data with Unsupervised & Unstrcutured data
- Generative AI (Generates something new based on given problem)
- NLP Tasks (Ex: Classification, named-entity recognition)
- Tune Foundation Models
- Prompt Engineering (Ex: Does this sentence have a positive sentiment or negative sentiment?)
Advantage of Foundation Models
1. Performance (TB's of data is applied to small tasks)
2. Productivity (You need far less label data to get to task-specific model)
Disadvantages of Foundation Models
1. Compute (Expensive to train, Running inference)
2. Trust (Unstructure data, data is scraped from the Internet)
Example of Foundation Models
- LLMs (Ex: ChatGPT)
- Vision (Ex: DALL·E 2)
- Code (Ex: Copilot)
- Chemistry (Ex: molformer)
- Climate (Ex: Earth Science Foundation models using geospatial data)