What Is an AI Engineer?

Over the past year, a new developer persona has emerged: the AI engineer. This role differs from both traditional software engineers or data science/ML engineering. It increasingly looks like the AI engineer's task is centered around productizing AI and making it consumable through REST APIs. Let's dive into the key characteristics and responsibilities of an AI engineer.

This post primarily contains summaries and observations based on the original AI engineer manifesto and recent market developments.

Productizing AI

AI engineers are all about productizing AI advances. They take on the challenge of trying out, applying, and turning GenAI capabilities of LLMs into real products. This involves working with product-specific data, conducting pilot projects, and ensuring that the AI models are production-ready. Unlike researchers, AI engineers focus on the practical aspects of shipping AI products. The commoditization of LLM technology has enabled engineers to accomplish a wide range of AI tasks that would have previously required an entire research team.

Tooling and Skills

AI engineers are well-versed in heterogeneous development environments. They come from a software engineering background and aren't Python-only, but rather "full-stack," using a variety of open-source tools and frameworks to move faster. They tend to have a very product-centric mindset, thinking along the lines of "What can I do with this new LLM, and how?" rather than getting caught up in the scientific and research-related details of the model itself.

Prompt Engineering as the New API

AI engineers think in terms of APIs, leveraging GenAI through API-driven architectures. Arguably, prompt engineering has become part of this API mindset. Prompt engineering is the native interface for interacting with LLMs, typically through a REST API to allow remote access and standardized communication. AI engineers need to be aware of the specifics of prompt engineering, per LLM. This is essential to be able to use a given LLM successfully. Much like with common API tools, a product-focused AI engineer "can prompt an LLM, and build/validate a product idea."

Organizational Structure

AI engineers should be part of AI teams. However, many organizations still have AI teams siloed within individual business units, while centralized AI teams are still rare. The expansion of AI techniques beyond traditional machine learning has led to a reciprocal blending of roles, with data scientists, ML engineers, and AI engineers increasingly sharing overlapping responsibilities and skill sets. The spectrum (or continuum) of roles and tasks described in the original post is still very useful when thinking about team composition: Deciding who takes on what responsibilities and who needs what to be successful depends on individual skill sets, not job titles. Overall, AI teams have begun to play a crucial role in transitioning AI applications from pilot to production.

Challenges and Expectations

AI engineers face unique challenges compared to traditional software engineers. LLMs can be unpredictable, and their output can be potentially dangerous. Modern product teams often have unrealistic expectations for high accuracy and alignment right after launching an LLM product, but AI engineering requires a commitment to a process of continuous evaluation and monitoring. Similarly, data science and ML engineering team members must learn and commit to modern software engineering practices.

The Future of AI Engineering

As the field of AI continues to evolve, the role of the AI engineers will become increasingly important. They will help bridge the gap between AI/ML research, data engineering, data science and practical business applications. AI teams will continue to focus on tasks like implementing production-ready RAG, as well as the data integration, processing, and ingestion.

Overall, the rise of the AI engineer marks an exciting shift in the world of software development. By embracing this new persona and its unique skill set, organizations can harness the power of GenAI to actually build useful products.

Learn More

There have been a lot of conversations lately about the role of the AI engineer. Here are some notable examples.

Tom Tunguz wrote about his observations in 2023 (and keeps reflecting on this topic):

• Top 10 Trends for Data in 2024
• The Paradox of AI and Data Roles: How Automation Will Increase Demand for Data Professionals
• How ChatGPT Will Restructure Engineering Teams & Create Opportunity for Startups

Shreya Shankar has recently written about the tensions and the challenges of an AI team:

• Short Musings on AI Engineering and "Failed AI Projects"