Picture the help button in a shopping app. A customer types that a parcel never arrived. Behind the button, a GenAI engineer wrote the service that looks up the order, fetches the return policy from the company's own documents, and hands both to the model with careful instructions. Those documents had been cut into small pieces and stored in a vector database such as Pinecone or pgvector, a store that finds text by its meaning rather than its exact words, so the right paragraph turns up in a moment. The engineer also wrote the rules that stop the assistant from promising a refund the company will not give, and the tests that catch it when it makes something up. When a question is beyond it, the assistant hands the chat to a human agent, because the engineer built that path too.

The work mixes ordinary backend building with work that exists only because of the LLM, the large language model. On the shopping app that could mean a new tool that lets the assistant check delivery status by itself, which turns it into what engineers call an agent, a model that takes actions rather than only replying. It could mean rewriting the prompt, the instructions sent to the model with every request, after the assistant began answering in the wrong tone. It also means reading failed conversations to see where retrieval, the step that finds the right documents before the model answers, picked the wrong one, and fixing it. The code is usually Python, with frameworks like LangChain, LangGraph or LlamaIndex, served through FastAPI. Before any of this is built, the engineer agrees with product managers and designers what the assistant should and should not do.

Building the feature is only part of the job. Around it sits a set of duties that every GenAI engineer shares.

Testing runs through all of it. A model can answer the same question differently on different days, so the engineer keeps a set of sample questions, scores every change against them, and treats a drop in the score the way other engineers treat a failing test. Every change is also read by a teammate in a code review and shipped through a CI/CD pipeline like any other service.

Once the assistant is live, the engineer watches it. Each call to the model costs money and takes time, so the engineer tracks what every answer costs, how long it takes and how often people ask for a human instead. Guardrails, the rules that stop the assistant from leaking private data or saying something harmful, need regular review as people find new ways to trick it. Providers release new versions of their models often, and a switch can change both cost and behaviour, so the engineer keeps up with them and tests each one before moving to it.

Planning is part of the job as well. Work is usually split into short cycles called sprints and tracked as tickets in a tool such as Jira, and much of the talk with product teams is about what the model can be trusted to do on its own.

A GenAI engineer does not train the model. The AI model engineer does that, training, fine-tuning and serving large models and making them run fast. A machine learning engineer builds a smaller model that predicts one thing from a company's own data, such as fraud or the coming month's sales. The GenAI engineer sits closer to a backend engineer who has learned to treat a model as one more service, one that answers in words and is sometimes wrong.

The ideal candidate likes building whole products, can live with some uncertainty, and enjoys the puzzle of getting steady behaviour from something that is not fully predictable. People often arrive from backend or full-stack work and pick up the model side along the way. Clear writing helps more than one might expect, since a prompt is a piece of writing.

With experience, the work widens from one feature to a whole AI system. A senior engineer decides which models to use, how agents pass work to each other, how answers are checked and what each answer costs. Some move toward fine-tuning and model work, and some lead the teams that decide where AI belongs inside a product.