Python is the language of this role, named in most job posts, because every LLM library and every model provider's SDK is written for it first. Java is second, and TypeScript and Node.js third, mostly on the software-engineer side. The application server that wraps the model is FastAPI when it is named, because it handles many slow outside calls at once and describes its own API, with Flask and Django in a few job posts and Spring Boot where the product is a Java system. The server does the same job it does in any backend: it accepts a request, works out what is being asked, and sends back a response. The difference is that building the response now means asking a model.

At the heart of the work is a call to a large language model. The request carries a prompt, the instructions and the question, and whatever context the model needs to answer, and the response comes back as text that the server has to check, shape and store. The OpenAI API is the provider named most, followed by Anthropic's Claude, then Azure OpenAI, which is OpenAI's models sold through Microsoft, and Gemini through Vertex AI on Google's cloud. Amazon Bedrock and Azure AI Foundry are the clouds' own platforms for reaching several models through one door, and together they are tagged as enterprise AI platforms on a fair share of job posts, most at the GCCs, the information companies and the consulting arms, where the company has already chosen a cloud and wants its models from the same place. HuggingFace and LLaMA appear where a team runs an open model itself, and Mistral AI in a handful.

Prompt engineering, the craft of writing the instructions so the model answers reliably, is assumed in every job post rather than listed as a tool. Guardrails AI appears where the output has to be checked against rules before it reaches a user, and Responsible AI and AI Governance are tagged on a fair share of job posts, most at the GCCs of the banks and insurers and in the consulting arms, where a wrong answer has regulatory consequences.

One call to a model rarely finishes a job. The server has to fetch data, call the model, read the answer, decide what to do next, perhaps call a tool or another model, and keep going until the task is done. The libraries that organise this are the most-named skills in the role after Python itself. LangChain leads by a wide margin, and it is asked for in a large share of the LLM application builder job posts. LangGraph, from the same team, is for agents whose steps branch and loop rather than run in a line, and it is second. LlamaIndex is for connecting a model to a company's documents and data. Semantic Kernel is Microsoft's equivalent, and it appears mostly at Accenture and in Azure work. CrewAI, AutoGen and ADK are frameworks for several agents working together, and LangChain Agents is the agent layer of LangChain.

An agent is a program that gives a model a goal and a set of tools and lets it decide which to use. MCP, the Model Context Protocol, is the new standard for describing those tools so any model can use them, and A2A does the same for agents talking to other agents. Both appeared in job posts this year, most at the information companies and the GCCs, and they are the clearest sign of a team building agents rather than assistants. LangSmith and Langfuse are the tools for watching what a chain or agent actually did on each run, which is how these systems are debugged.

A model knows nothing about a company's own contracts, claims, filings or support tickets. Retrieval-augmented generation, usually called RAG, fixes that: the documents are cut into pieces, each piece is turned into a list of numbers called an embedding that captures its meaning, the embeddings are stored, and when a question comes in the server finds the pieces closest in meaning and hands them to the model with the question. This is the single most common thing a GenAI engineer builds, and it is the whole business at the information companies that sell news, legal, financial and market-research content.

The store for embeddings is a vector database, and job posts name several. Pinecone is the most common, followed by Weaviate, pgvector, which adds vector search to PostgreSQL so a team can keep one database, Milvus, FAISS, ChromaDB and Qdrant. Vector Search, Semantic Search and Information Retrieval are named as skills in their own right, most at the information companies and in big tech. Elasticsearch appears where keyword search and vector search are combined, and Knowledge Graphs where the relationships between documents matter as much as their content. The database companies themselves, which are adding vector search to their products, hire for this work close to the data platform.

The model call is one step in a system that otherwise looks like any backend, and the backend and data layer is the most common extra ask in the whole role, on most job posts. Data Pipelines and ETL are named where documents and records have to be collected, cleaned and refreshed before they can be embedded, most at the information companies and the GCCs. MongoDB, Redis, PostgreSQL, DynamoDB and Elasticsearch appear as the ordinary stores behind the application, with Redis doubling as the cache that stops the same question being sent to the model twice. Pandas and NumPy appear where the engineer also prepares data. The REST API that the product's screens call is assumed in every job post, and Spring Boot, FastAPI and NestJS are what builds it.

Most GenAI engineers never train a model, but a fair share of job posts ask for enough model knowledge to fine-tune one or evaluate several. TensorFlow and PyTorch are named in a minority of job posts, with Transformers, Scikit-learn and MLflow behind them, and SageMaker and Kubeflow where models are trained and served on the cloud. These appear most at the information companies and the GCCs, where a team may fine-tune a model on its own documents, and at the AI product companies. In the ordinary GenAI job they are a plus.

A small set of job posts, mostly in big tech, build systems that listen and speak. Spoken Conversational AI, Speech Processing, NLU and Speech LLMs are the skills, with Sentiment Analysis and Conversational Analytics for understanding what was said. This is a specialism of its own and sits closest to the voice assistants of the large platforms and the contact-centre companies.

A fair share of job posts want the engineer to build the interface of the AI product too, and on the software-engineer side it is most of them. React with TypeScript leads, with Angular and Vue close behind, which is unusual, and almost entirely because Accenture's job posts list all three. Node.js and NestJS appear where the whole product is in TypeScript. Frontend work is asked for most at Accenture and the staffing platforms, and least at the information companies and the GCCs, where the AI engineer builds the service and someone else builds the screen.

The service that calls the model still has to run for many users at once, behave the same in testing and in production, and keep going when a machine fails, and here the cloud also decides which models are within reach. Cloud and containers are asked for in a large share of job posts, and in most at the product companies, the information companies and the GCCs. AWS is the most-named cloud, followed by Azure, which is where Azure OpenAI and Semantic Kernel work lives, and then GCP, which is where Vertex AI and Gemini live. Kubernetes and Docker are named in a fair share, Kubernetes a little ahead, and Terraform and CloudFormation where the engineer describes the infrastructure as code. Containers are asked for most at the AI product companies and least at Accenture.

A GenAI product changes faster than most, because models, prompts and libraries all change under it. A build-and-release pipeline runs the tests on every change, packs the service into a container and rolls it out, and in this role it also has to re-run the evaluation that checks the model's answers have not got worse. GitHub Actions is the pipeline named most, with Azure DevOps and Jenkins behind it. Pipelines are asked for in a good share of job posts, most in the consulting arms, the GCCs and big tech.

Testing a GenAI system is harder than testing ordinary code, because the same question can get a different answer each time. Job posts assume ordinary unit testing and rarely name a tool, and the evaluation of model output is described in the text of the post rather than as a skill. LangSmith and Langfuse are the closest thing to a named tool for it. Underneath all of it sits the ordinary craft of building software in a team: Git for source control, pull requests and reviews, issue tracking, and a rhythm of small, frequent releases.

A GenAI engineer who writes Python well, can wrap a model call in a FastAPI service, knows LangChain and LangGraph, has built a retrieval system over documents with a vector database such as Pinecone or pgvector, has used the OpenAI API and at least one other provider, and can ship in a container on AWS or Azure through a GitHub Actions pipeline, meets the core of nearly every LLM application builder job post. An engineer on the software side needs the same understanding of the model layer with a stronger backend and frontend, in Java and Spring Boot or TypeScript and React. The variations belong to the employer. The business software companies want agents inside their finance, HR and service products, with AWS, Kubernetes and LangChain. The information companies want retrieval above all, with Vector Search, Data Pipelines, MCP and enterprise AI platforms. The GCCs of banks, insurers and healthcare groups want the same with Responsible AI and governance, and the most pipelines. Big tech builds the platforms themselves and asks for Information Retrieval and voice. The consulting arms want engineers who can build on all three clouds. Accenture, the widest door by far, wants an engineer who can be placed on any stack, and lists them all.