Homework – 2 (GenAI)

· Go to OpenAI.com

· Sign up for an account, launch GPT-3 playground

· Brainstorm a new task

Perform. a single “traditional” NLP Task

LLMs have shown that they are extremely capable at performing (what are now) seen as relatively straightforward tasks. As shown in the slides, some of these are:

· Grammar correction

· Summarization

· Sentiment prediction

Newer models have shown that they can perform. more complex tasks such as:

· Step by step reasoning

· Joke explanation

To get familiar with the playground, perform. one of these tasks.

· Does the output make sense?

· Are you able to create any adversarial examples that cause the model to fail?

Brainstorm new tasks

The real power of these models lies in their new capabilities. What are some brand new tasks that can test these models? As you brainstorm your tasks, also consider the following questions:

· How would you evaluate the model’s performance on your task?

· Are there specific cases where the model “fails” on your task, and why would that be?

· What types of systems could you see this task being embedded in, and what are the system level implications of that?