GenAI 新范式：GraphRAG NebulaGraph Team的探索与实战

GenAI新范式：GraphRAG NebulaGraphTeam的探索与实战 目录 01 图Graph是什么 02 03 04 为什么需要GraphGraphRAGGraphRAG案例 图Graph是什么 什么是图 Ingraphtheory,agraphisadatastructureconsistingofnodes(vertices)andedges,usedtorepresentrelationshipsbetweenentities,withnodesrepresentingobjectsandedgesdenotingconnectionsorassociationsbetweenthem. ThepaperwrittenbyLeonhardEulerontheSevenBridgesofKönigsbergandpublishedin1736isregardedasthefirstpaperinthehistoryofgraphtheory. -- https://en.wikipedia.org/wiki/Graph_theory Author:WeyGu Source:https://github.com/wey-gu/graph-vis 什么是图 Ingraphtheory,agraphisadatastructureconsistingofnodes(vertices)andedges,usedtorepresentrelationshipsbetweenentities,withnodesrepresentingobjectsandedgesdenotingconnectionsorassociationsbetweenthem. ThepaperwrittenbyLeonhardEulerontheSevenBridgesofKönigsbergandpublishedin1736isregardedasthefirstpaperinthehistoryofgraphtheory. -- https://en.wikipedia.org/wiki/Graph_theory Author:WeyGu Source:https://github.com/wey-gu/graph-vis 为什么需要Graph WhyGraph -KnowledgeGraph GraphUseCases -KnowledgeGraph -RecommendationSystem Author:WeyGu Source:https://www.siwei.io/recommendation-system-with-graphdb GraphUseCases -KnowledgeGraph -RecommendationSystem -SocialNetwork Author:WeyGu Source:https://siwei.io/en/nebulagraph-sns/ GraphUseCases -KnowledgeGraph -RecommendationSystem -SocialNetwork -DevSecOps Author:WeyGu Source:https://github.com/wey-gu/k8s-graph GraphUseCases -KnowledgeGraph -RecommendationSystem -SocialNetwork -DevSecOps -FraudDetection Author:WeyGu Source:https://siwei.io/en/fraud-detection-with-nebulagraph GraphUseCases -KnowledgeGraph -RecommendationSystem -SocialNetwork -DevSecOps -FraudDetection -SupplyChainManagement Author:WeyGu Source:https://github.com/wey-gu/supplychain-dataset-gen GraphUseCases -KnowledgeGraph -RecommendationSystem -SocialNetwork -DevSecOps -FraudDetection -SupplyChainManagement -LLM Author:WeyGu Source:https://bit.ly/graph-rag-workshop GraphUseCases -KnowledgeGraph -RecommendationSystem -SocialNetwork -DevSecOps -FraudDetection -SupplyChainManagement -LLM -DigitalMarketing Author:WeyGu Source:https://www.siwei.io/demo-dumps/telco_biz/Subscriber_Profile_Linkage.html GraphRAG RAG：开卷考试 Retrieval翻参考资料 Augmented上下文学习增强领域知识 Generation现学现卖(LLM) Retrieval翻参考资料 Augmented上下文学习增强领域知识 Generation现学现卖 RAG：开卷考试范式 索引期 分割、嵌入 分割 嵌入 数据分割、创建语义向量表达并存储 查询期 根据任务语义，寻找相似参考 -GetEmbeddingofTaskDescription -SemanticSearchTop-KinsameEmbeddingVectorSpaceforRelatedChunksofContext 大模型学习上下文，尝试推理得出答案 -In-contextlearningtosynthesizeanswersorreacttothenextmove 嵌入 Retrieval翻参考资料 Augmented上下文学习增强领域知识 Generation现学现卖 分割的挑战：海里捞针（Needleinahaystack） -分割索引很难召回细粒度知识点，分块尺寸是一个很强的信息浓度假设，可能会稀释块内的知识。 -线性分割是对知识内在的全局上下文、关联的丢失。 -Splitcanmakeitchallengingtoretrievefine-grained,fragmentedknowledge.Partitioningmaydilutecertainpiecesofknowledgeacrossvarioussegments. -Linearlysegmentinglargechunksofknowledge canleadtothelossofglobalcontext/ interrelationship. Source:https://www.siwei.io/graph-enabled-llama-index/ 嵌入的挑战：开卷考试相关不是相似Relevancevs.Similarity -语义是领域知识的一部分，基于通识构建的通用Embedding模型并不了解。 关于保温杯的文章 ImageAuthor:DALL-E2&古思为 Source:https://bit.ly/graph-rag-workshop 关于保温大棚的文章 DomainKnowledgenotawareofbyvanillaEmbeddingModels 图帮助做好开卷考试：GraphRAG -幻觉：图上可以做确定可复现的高质量关联，非概率性、易幻觉关联 -成本：图库是一种高效、廉价的推理与知识合并的实现：淬炼领域知识推理、转变为I/O查 -效果：图索引是凝练、细粒度、保有全局互联结构的压缩：大海捞针、穿针引线 -KnowledgeGraphis_ofKnowledge: -RefinedandConciseForm -Fine-grainedSegmentation -Interconnected-structured -Knowledgein(existing)KGisAccurate -QuerytowardsKGisStable,Deterministic -DomainKnowledgewaspusheddowntoKGinrelationships,whichenabledreasoninginquery(andcheaplypersisteddomainknowledge) Author:SiweiGU Source:https://siwei.io/graph-enabled-llama-index/harry_potter_graph.html GraphRAG：图结构数据/知识参与辅助的RAG方法 -幻觉：图上可以做确定可复现的高质量关联，非概率性、易幻觉关联 -成本：图库是一种高效、廉价的推理与知识合并的实现：淬炼领域知识推理、转变为I/O查 -效果：图索引是凝练、细粒度、保有全局互联结构的压缩：大海捞针、穿针引线 -KnowledgeGraphis_ofKnowledge: -RefinedandConciseForm -Fine-grainedSegmentation -Interconnected-structured -Knowledgein(existing)KGisAccurate -QuerytowardsKGisStable,Deterministic -DomainKnowledgewaspusheddowntoKGinrelationships,whichenabledreasoninginquery(andcheaplypersisteddomainknowledge) Author:SiweiGU Source:https://siwei.io/graph-enabled-llama-index/harry_potter_graph.html GraphRAG案例 例子：可解释性 图帮助做好开卷考试：GraphRAG[演示1简单演示] 图上可以做确定可复现的高质量关联，非概率性、易幻觉关联 图帮助做好开卷考试：GraphRAG[演示2相关与相似] - 例子：降低Embedding-RetrievalHallucination -图库是一种高效、廉价的推理与知识合并的实现：淬炼领域知识推理、转变为I/O查 -图索引是凝练、细粒度、保有全局互联结构的压缩：大海捞针、穿针引线 Author:SiweiGU Source:https://www.siwei.io/demo-dumps/local-llm/Graph_RAG_Local.html 例子：降低Embedding-RetrievalHallucination 图帮助做好开卷考试：GraphRAG[演示3相关与相似] 幻觉：图上可以做确定可复现的高质量关联，非概率性、易幻觉关联 - -成本：图库是一种高效、廉价的推理与知识合并的实现：淬炼领域知识推理、转变为I/O查 -效果：图索引是凝练、细粒度、保有全局互联结构的压缩：大海捞针、穿针引线 提问主角之间不存在的剧情 GraphRAG消弭幻觉: -向量搜索 -图探索 -rerank Author:SiweiGU Source:https://bit.ly/graph-rag-workshop C乐op知yr乐igh享t©/2同02心4b共y济Hangz知ho行u合Yu一esh/u不Te负ch所no托logy,Inc.Allrightsreserved.ConfidentialInformation. 图帮助做好开卷考试：GraphRAG[演示4精细知识检索]