Top 10 — Must Know GenAI with LLM [ Large Language Model] Interview questions

Amber Ivanna Trujillo
10 min readOct 20, 2023

With the rise of the LLM era, every company and organization is looking into resources to learn and implement the best features of this technology. The hiring efforts across all the major sectors including healthcare, government, transportation, finance, education and Technology is beyond the ceiling.

NOTE — See the bonus question towards the end :).

Below we represent top 10 LLM [Large Language Model — based] must know / must ask questions to gauge the performance of a candidate.

1. Which event and publication in the history of Natural Language Processing [NLP] shifted gears towards the current trend in LLMs?

Google published “Attention is all you need” on 06-2017 in NeurIPS, and introduced the Transformers architecture. This laid the foundation of a revolution and boom for most LLM and Deep Learning architectures.

  • The Paper laid out a number of reasonably COMPLEX DATA PROCESSES INSIDE THE TRANSFORMER ARCHITECTURE which is responsible for helping the model learn the relationship of all the tokens within a language
  • Attention mechanism has been around for a while, but it started to make more impact because of the Massively Parallel way of Computing Power offered by Modern GPUs [ By NVIDIA, AMD etc] and GPU clusters.
  • Not only did Text or NLP benefitted from Transformer’s attention mechanism, also Vision transformers got improved.
  • Self-Attention and Multi-Headed Self-Attention Mechanism worked by creating Attention Maps of each incoming Token through its Encoder and Decoder layers. Attention Maps learn the association each token with every other token in the context of language.
  • LLMs built using MHSAM — Multi-Headed Self-Attention Mechanism, have great language understanding. LLMs learned the statistical patterns, relationships, and contextual nuances present in the language.

2. What is the concept behind GPT and what does it stand for? Describe a few details on this model and its evolutionary roadmap.



Amber Ivanna Trujillo

I write about Technical stuff, interview questions, and finance Deep Learning, LLM, Startup, Influencer, Executive, Data Science Manager