Unavailable

This livestream is restricted

Already a member? Login with your membership email address

The Fifth Elephant For members

RWKV: reinventing RNNs for the transformer era

The Fifth Elephant paper reading meet-up - April 2024

Tickets

Loading…

About the Paper

In the last three years, RNNs have caught up with the unparalleled capabilities of Transformers. The promise of Receptive Weighted Key Value (RWKV) is that this novel architecture combines the desirable aspects of both RNNs and Transformers: the massively parallelizable transformer-esque training, and RNN’s consistent computational and memory complexity during inference.
RWKV (pronounced as “RwaKuv”) is an attention-free language model, theoretically capable of handling an “infinite” context length.

Key takeaways for audience

  • Intuitive understanding of RWKV’s formulation, using math and code.
  • Information about performance benchmarks and the scaling laws - and evaluate how you can use RMKV in your work.
  • A demo of RWKV’s inference prowess.

About the presenter

Yashodeep Deshmukh is Deputy Manager at Ashok Leyland.

RSVP

This paper discussion will be held at Atlassian’s office in EGL, Bangalore. In-person attendance is free. The Fifth Elephant members can join remotely to watch the live stream.

About The Fifth Elephant monthly paper discussions

The monthly discussions are organized to understand popular papers in Generative AI, DL, and ML domains. Papers are curated to benefit the community. The paper discussion is organized on the first Friday of each month, from 5:30 PM - 7:00 PM.

How you can contribute

  1. Suggest a paper at https://hasgeek.com/fifthelephant/call-for-papers/sub
  2. Moderate/discuss a paper someone else is proposing.
  3. Pick up a membership to support The Fifth Elephant’s activities.
  4. Spread the word among colleagues and friends. Join The Fifth Elephant Telegram group or WhatsApp group.

Contact

For inquiries, leave a comment or call The Fifth Elephant at +91-7676332020.

Hosted by

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more

Supported by

Venue host

Atlassian unleashes the potential of every team. Our agile & DevOps, IT service management and work management software helps teams organize, discuss, and complete shared work. The majority of the Fortune 500 and over 300,000 companies of all sizes worldwide - including NASA, Audi, Kiva, Deutsche B… more

{{ gettext('Draft') }}

Hosted by

The Fifth Elephant - known as one of the best data science and Machine Learning conference in Asia - has transitioned into a year-round forum for conversations about data and ML engineering; data science in production; data security and privacy practices. more

Supported by

Venue host

Atlassian unleashes the potential of every team. Our agile & DevOps, IT service management and work management software helps teams organize, discuss, and complete shared work. The majority of the Fortune 500 and over 300,000 companies of all sizes worldwide - including NASA, Audi, Kiva, Deutsche B… more