Staff Software Engineer, LLMs
AI Tools
Agentic Frameworks
Tech Stack
Agent Workflow
Design and own user-facing LLM and generative AI features at Upstart. Build RAG systems, vector search, prompt design and model evaluation pipelines, and production APIs integrating LangChain/LlamaIndex and OpenAI into lending products.
About the Role
Upstart is seeking a Staff Software Engineer to develop foundational infrastructure for generative AI across the company's product teams through the Applied LLM team.
Key Responsibilities
- Design and build user-facing ML features that harness LLMs and generative AI
- Partner with product, design, and ML research to prototype intelligent experiences
- Own technical architecture for applied ML systems, balancing performance considerations
- Build scalable services and APIs for model outputs
- Collaborate across platform, infrastructure, and compliance teams
- Establish best practices for prompt design and model evaluation
Required Qualifications
- 6+ years software engineering experience with 2+ years on ML-driven products
- Proven leadership across engineering, product, and research stakeholders
- Strong backend development (Python, FastAPI/Flask)
- Cloud-native tooling expertise (Kubernetes, Docker, Terraform)
- Production LLM/ML model integration experience
- Excellent communication and collaborative approach
Preferred Qualifications
- GenAI/LLM framework experience (LangChain, LlamaIndex, OpenAI APIs)
- RAG, vector search, and real-time inference knowledge
- Full-stack development capabilities
- Prompt engineering and model evaluation expertise
- Compliance and explainability experience
- Track record influencing architecture at team/org level
Compensation & Benefits
US Remote: $199,500-$276,000 USD base
Additional benefits include bonuses, equity, 401(k) matching ($2:$1 up to $15,000/year), medical/dental/vision (90-100% covered), parental leave, wellness allowance, productivity allowance.
Location & Travel
Remote (US/Canada), operating on East/West Coast time zones. Quarterly on-site collaboration (3-4 days per quarter); Upstart covers travel expenses.
