RunwayML Week 1 Class Notes

Taught by Derrick Schultz & TA Lia Coleman

🔗 Links

• Class Slides

💥 This class's objectives

• Learn through making stuff.
• Understand basic machine learning (ML) concepts— Having a basic understanding of ML will help you know what you're doing in Runway!

👩‍🏫 Class Expectations

• Expect 3-6 hours of homework every week.
• Different ways to approach this class:
  1. Follow along each week, do every homework assignment, OR
  2. Follow along each week, work on one big project, OR
  3. Soak it in, ignore homework (don't recommend, but!)

🖼 AttnGAN demo

• Demir observation: The image generated as the sentence was typed in.
• Takeaway: This model sucks!! It doesn't do well on abstract sentences. It only works well on sentences it was trained on.
  • "the car was parked by the beach" is the best it'll do, pretty much. 💩
• Most state-of-the-art models do not produce good images off the bat.
• How was it trained?
  • It was trained on birds, dogs, cars, beaches.
  • The dataset it got was: People got a bunch of images, then they captioned them. This is what it was trained on.
  • In testing, we are doing the reverse: inputting a sentence to get an image.

🤔Moises: How is it working under the hood?