2025企业生成式AI峰会报告-Day2

Scaling AI driven Marketplace Products Dilip Patel, AI Product Lead @ Uber, ex-Amazon About Me & My Journey ●Data Scientist Turned Product Leader ○10+ years in scaling AI driven Search & Discovery, Monetization (Pricing,Promotions, Ads), Growth and Risk Products in Marketplaces/eCommerce○BigTech (Uber, Amazon, Salesforce) + SmallTech/Startups (Groupon,Fractal.ai, WeWork)○5+ Years in Data Science and Quant Finance & Risk at Wall Street banks ●When no working …. ○Student always: MBA, UC Berkeley; CS undergrad, Mumbai Univ; CFA○Coach AI PM UC Berkeley and AI Product Advisor @ Startup Accelerators○Outdoor: Ski, Water sports, Hiking, Biking, Tennis, Pickleball○Live with the Family in the SF Bay Area What We’ll Cover ●A 5‑pillar framework to scale AI in Marketplaces●Case studies●Org design, governance & trust●Takeaways●Q&A The Marketplace Reality ●Multi‑sided dynamics: buyers, merchants, earners, platform. ●Trade‑offs: GMV↑vs. margin/take rate, price competitiveness vs. P&L,speed vs. governance. ●Implication: You don’t ship a model—you ship a business outcome underconstraints. The 5 Pillars (Framework Overview) ●Start Practical— prototype fast on APIs; scale with OSS / in‑house ●Data Moat— exploit proprietary interaction data●Human‑in‑the‑Loop— augment experts; capture overrides●Optimize to KPIs— objective functions > features●Trust & Control— explainability, guardrails, fallback UX Pillar 1: Start Practical, Scale Smart ●Phase 1 (Weeks):Prove value with 3P APIs (OpenAI/Vertex) + thin glue. ●Phase 2 (Quarters):Migrate to OSS (Llama/Mistral) or bespoke for cost,latency, control. ●Case:Amazon new‑item pricing→started with ML benchmarking; successjustifiedRL enginefor price discovery & optimization. Pillar 2: Your Proprietary Data Moat ●Moat ≠ model.It’s clickstreams, comp signals, seller inventory, fulfillmentpatterns, UGC, returns. ●E.g.UGC and Contextual features embedding→conversionlift ●Prompt:What signals emerge only from your buyer–seller interactions? Pillar 3: Human‑in‑the‑Loop (HITL) ●System design:AI proposes; humans dispose. Every override is labeledfeedback. ●Pricing risk tools:AI suggested;Ops controls eligibility(based on SOP)→adoption↑because trust↑. Pillar 4: Optimize to Business KPIs ●Objective functions:Maximize GMV/LTFCF subject to competitiveness; notjust MAPE/precision. ●RL pricing:Optimized LTFCF with competitiveness guardrails→$XXXMMannualized FCF uplift, while keeping GMV stable. Pillar 5: Trust & Control Architecture ●Explainability: “Price moved −6% due to competitor −10% & elastic cohort.” ●Risk: simple reason codes; appeal flows. User controls: price floors, blacklist,exploration budgets; risk thresholds by cohort & geography. ●Fallbacks: when signals are weak→conservative policies; fail‑safe UX forpayments & marketplace integrity. ●Outcome: Trust→adoption→impact across buyers, sellers, and platform. Case-Study 1: Product Matching ●Problem:Heuristic/NLP matching struggled at 3P scale; high ops cost. ●Approach:Distilled LLM1 for product summaries; fine‑tuned LLM2 for matchdecisions. ●Results:>99% precision at 40% recall across the marketplace; ~30% of manualmappings automated for target marketplaces. Case-Study 2: Quantity Understanding (QU) ●Problem:Catalog quantity/pack hierarchies (pallet→case→bottle)ambiguous; structured‑only methods missed context. ●Approach:Prompted LLM builds a unit tree per ASIN (target unit, aspect,parent, units per parent, totals); feedback loop with ops labels; M5embeddings for similarity. ●Results:PSU accuracy 83%→97%; count accuracy 98%→99%. ●Services:simulation, RL‑MAB for discounts, feature engineering, self‑serveanalytics. Scaling the Team: The AI Product Pod ●Structure:Create full-stack product pods for each AI-driven area (e.g., Pricing Pod,Search Pod). ●Pods Own KPIs: GMV, Conversion, profit margins ●The AI PM Role is Different:○It's less about UI specs and more about owning theobjective function, the evaluation framework, and thebusiness KPIs.○They must understand model tradeoffs (precision vs. recall), experiment design,and how to build user trust. ○Instead of: "Build a promotions feature."○Think: "Design a system that maximizes conversion, subject to marginconstraints, while ensuring fairness to all participating sellers." Safety, Governance & Guardrails ●Objective integrity:LTV‑first, customer-trust‑bounded●Exploration budgets:rate limits, elastic cohorts, rollback triggers●Fairness & transparency:stability controls, justification snippets, audittrails●GenAI controls:RAG boundaries, policy prompting, red‑team evals, PIIhygiene Key Challenges in Marketplace AI ●Exploration vs Exploitation ○RL requires balancing discovery (new sellers/items) vs maximizing immediateconversion ●Defining ROI○Measuring GenAI shopping assistants beyond engagement→incremental GMV& margin ○Lower price→boosts conversion, but impacts seller P&L○Promotions→increase urgency, but risk cannibalization ○Black-box systems = lo