Menene RAG kuma Me yasa Ya zama dole?

Riba:

• Bayyana cewa RAG yana shigar da mahallin mahallin ba tare da canza ma'aunin samfurin ba kuma yana aiki tare da dabarun 'budewar jarrabawa'
• Kwatanta RAG tare da daidaitawa mai kyau da dogayen hanyoyin mahallin dangane da farashi, lokaci da yanayin amfani
• Lissafin matakan bututun RAG na yau da kullun wanda ya ƙunshi fihirisa da matakan tambaya

Komai karfin samfurin harshe (hankali na wucin gadi wanda ke fahimta da samar da rubutu; za mu kira shi samfuri a takaice daga yanzu), bai san kwangilar da kamfanin ku ya rattabawa hannu ba a jiya, shafin wiki na ciki (tushen ilimin cikin gida), ko bayanin sakin da aka buga a safiyar yau. Samfurin ya iyakance ga ilimin gabaɗaya har zuwa ranar da aka horar da shi; Ana kiran wannan "kwanakin yankewar ilimi". RAG (Retrieval-Augmented Generation) ya cika daidai wannan rata: ya nemo takardun kamfanin da suka shafi tambaya, ya ba shi samfurin a matsayin mahallin (wato, ƙarin rubutun da zai karanta yayin samar da amsa) kuma yana da amsar da aka samar bisa ga wannan mahallin.

A cikin wannan rukunin, za mu ga a sarari menene RAG, lokacin da aka fifita shi akan waɗanne hanyoyi, da matakan bututun RAG na yau da kullun. Duk raka'o'in da ke gaba za su zurfafa sassan wannan taswirar ɗaya bayan ɗaya.

Asalin Ra'ayin RAG: Buɗe Jarrabawar Littafi

Bari mu bayyana RAG a cikin jumla ɗaya: "Farko nemo takaddar da ta dace, sannan a sa ƙirar ta karanta waccan takarda kuma a buga amsar daidai."

Misali mafi fa'ida shine wannan: RAG yana motsa samfurin daga "jarrabawar littafi mai rufe" zuwa "jarrabawar budadden littafi." A cikin jarrabawar littafin da aka rufe, ɗalibin yana amsawa daga ƙwaƙwalwar ajiya kawai; Akwai babban haɗari na yin abin da ba ku tuna ba. A cikin budaddiyar jarrabawar littafi, dalibin ya amsa ta hanyar kallon tushen da aka ajiye a gabansa. A cikin RAG, samfurin baya amsawa daga ƙwaƙwalwar ajiyarsa, amma daga rubutu na yanzu da takamaiman ka ba shi.

Mahimmin mahimmanci: RAG ba ya canza ma'aunin samfurin, wato, biliyoyin ma'auni na lambobi waɗanda samfurin ya koya. Ba ku sake horar da ƙirar ba. Ga kowace tambaya, kuna cusa guntun rubutun da suka dace da waccan tambayar cikin gaggawar (rubutun umarni da aka aika zuwa samfurin). Don haka ba dole ba ne ka sake horar da samfurin lokacin da aka sabunta takarda; kawai kuna sabunta rikodin da ya dace a cikin bayanan bincike.

Alamomi: Tambayoyi biyu sun tantance ingancin RAG: (1) Shin kun sami takaddun da ya dace? (2) Shin samfurin ya karanta shi daidai? Na farko shine "kyakkyawan maidowa", na biyu shine "ingantattun tsara". Ana auna su biyu kuma an inganta su daban.

RAG, Kyakkyawan-Tuning ko Dogon Magana?

Hanyoyi uku galibi suna rikicewa yayin neman mafita ga matsalar kungiya. Mu fayyace sabaninsu. Kyakkyawan daidaitawa shine sabunta ma'aunin samfurin tare da bayananku da koya masa sabon hali/ salo. Dogon mahallin yana nufin cike duk takaddun kai tsaye cikin hanzari ba tare da wani zaɓi ba.

kusanci

Me yake aikatawa

Yaushe ya dace?

Farashin / Hatsari

RAG

Injects daftarin aiki a matsayin mahallin

Canje-canje akai-akai, m, takamaiman bayani

Ƙananan; mai sauƙin sabuntawa, ana iya kawo tushen tushe

Kyawawan daidaitawa

Yana sabunta ma'auni tare da sababbin bayanai

Kafaffen salo/tsara/koyarwar harshe

Maɗaukaki; Ana buƙatar sake horarwa tare da kowane sabuntawa

Dogon mahallin kawai

Cika duk takardu cikin hanzari

Karami, saitin takaddun da ke tsaye

Farashin alama da haɗarin "rasa ɓangaren tsakiya" yana ƙaruwa

A matsayinka na mai mulki: Kyakkyawan daidaitawa yana koya wa samfurin yadda ake magana; RAG yana gaya wa ƙirar abin da ya sani. A yawancin yanayin kasuwancin, RAG ana gwada shi da farko saboda yana da arha, ana iya sabuntawa, kuma yana iya nuna tushen amsar. Dogon mahallin yana da ma'ana idan tsarin daftarin aiki ya kasance ƙanƙanta kuma ƙayyadaddun (misali littafin jagora mai shafi 20 guda ɗaya); Amma tare da dubban shafuka, yana da tsada kuma samfurin na iya rasa bayanai a tsakiyar dogon rubutu.

Babban Bututun RAG

RAG ya ƙunshi manyan matakai guda biyu: ƙididdiga (shiri, yi sau ɗaya ko lokaci-lokaci) da kuma tambaya (yana gudana akan kowace tambayar mai amfani).

Fihirisar mataki-mataki (a layi, ba tare da jiran mai amfani ba):

1. Tattara: Jawo takardu daga tushe (PDF, wiki, tsarin tikiti, bayanai, imel).
2. Chunking: Rage dogon rubutu zuwa ƙananan sassa masu iya sarrafawa.
3. Embed: Maida kowane bangare zuwa sakawa (lambar vector mai ɗaukar ma'anar rubutu).
4. Ajiye: Rubuta ramummuka tare da rubutu da metadata (tushen, kwanan wata, bayanin izini) zuwa bayanan bayanan vector.

Tambaya ta mataki-mataki (kan layi, yayin da mai amfani ke jira):

1. Maida tambayar mai amfani zuwa sakawa.
2. Maido mafi kamanni sassa daga rumbun adana bayanai na vector.
3. Sanya waɗannan guda + tambaya cikin samfuri mai sauri.
4. Sami amsar mahallin da tushen sa daga samfurin.

# Ƙididdigar ƙa'idar lokaci na bincike (ba a dogara da harshe ba) tambaya = "Kwana nawa na hutun shekara?" question_vektor = embed(question)parts = vektor_db.search(question_vektor, top_k=4) # mafi kamanceceniya sassaprompt = f"" Amsa QUESTION ta amfani da CONTEXTI a ƙasa, idan bayanin bai kasance a ƙasa ba. Fitting.CONTEXT:{sassa}TAMBAYA: {tambaya}"""amsa = model.uret(gaggawa) # misali. samfurin: claude-opus-4-8

Wannan kwarara taswirar kowane mataki ne, wanda zamu kwashe daya bayan daya a cikin raka'o'in da ke gaba.

Rauni Mai Sauƙi / Ƙarfi Mai ƙarfi

Ko da tare da mahallin RAG iri ɗaya, ingancin saurin yana canza amsar.

Ƙarƙashin hanzari (buɗe don samfurin dacewa, baya buƙatar albarkatu):

Yi amfani da wannan bayanin kuma faɗi izinin shekara: {ɓangarorin}. Tambaya: {tambaya}

Ƙarfin faɗakarwa (ƙasa + "Ban sani ba" izini + buƙatar albarkatun):

Amsa kawai bisa ga CONTEXT na ƙasa. Idan babu cikakkiyar amsa a cikin mahallin, rubuta "Ba zan iya samun bayani game da wannan a cikin takardun ba"; Kar kayi tsammani. Ƙara alamar [Source: file_name] na guntun da kuke dogara da shi a ƙarshen amsar ku. BAYANI: {gudu} TAMBAYA: {tambaya}

Mini Cases guda uku

Case 1 - Mataimakin HR (Human Resources). Kamfanin yana da littafin jagora na HR mai shafuka 340 kuma ma'aikata suna yin matsakaicin tambayoyi 90 a rana. An yi ƙoƙarin daidaitawa, amma tun da an sabunta littafin a kowane wata, ana buƙatar sake horarwa kowane lokaci; Kudin ya kai dubban daloli a wata. Bayan canzawa zuwa RAG, an rage sabuntawa zuwa mataki na "sake-fididdigar daftarin aiki" (mintuna) kuma adadin amsa daidai ya karu daga 71% zuwa 93% a ma'aunin hannu.

Case 2 - Tallafin abokin ciniki. Ƙungiyar tallafi tana da tikiti 12,000 da aka warware da kuma labaran taimako 800. Yana ɗaukar matsakaicin mintuna 4 don wakili ya sami amsa da hannu. Lokacin da mataimaki na RAG ya kawo bayanan 5 mafi dacewa kuma ya samar da amsa daftarin aiki, an rage lokacin zuwa 40 seconds; Amma ƙungiyar ta fahimci haɗarin "kallon rashin tabbas ta hanyar kawo labarin da ba daidai ba" kuma ta sanya ambaton tushen ya zama tilas.

Harka 3 - Shari'a. Wata ƙungiyar kwangila ta tambayi "a cikin waɗanne kwangiloli ne batun sirri ya wuce shekaru 5?" Ya yi tambayar. A cikin gwaji mai tsawo, an cika kwangiloli 60 cikin hanzari guda; samfurin ya tsallake kwangilar na tsakiya biyu. Lokacin da kawai aka gabatar da abubuwan da suka dace tare da RAG, farashin alamar ya ragu da 80% kuma an sake saita tsallakewar da ta ɓace.

Me yasa ake buƙatar RAG?

• Yanzu: Kuna samun damar bayanai bayan ranar yanke horo.
• Bayani na musamman: Ba a haɗa takaddun ku na ciki a cikin horar da kowane samfuri; Kai kaɗai za ku iya bayarwa.
• Tabbatarwa: Kuna iya buga tushen amsar (citation) - mai mahimmanci don dubawa da amana.
• Ikon hallucination: Ya dogara da rubutun da aka sanya a gabansa maimakon yin samfuri.
• Farashin: Yana da arha da sauri don saka aiki fiye da daidaitawa.

Tsanaki: RAG ba sihiri bane. Idan kun kawo guntun da ba daidai ba, samfurin ya isa ga amsar da ba daidai ba yana kallon "aminci." Ka tuna kalmar "Kyauta maidowa = ingancin RAG".

Kuskuren gama gari

• Kuskure RAG don daidaitawa mai kyau: RAG baya canza ma'aunin nauyi; Yana ƙara mahallin kawai. Rikita waɗannan biyun zai haifar da zabar gine-ginen da ba daidai ba.
• Rashin ƙyale "Ban sani ba": Idan faɗakarwa ta bar samfurin kyauta don cike gurbin, zai gyara.
• Ba ambaton tushe: Amsa ba tare da tushe ba ba za a iya bincika ba; Mai amfani ba zai iya lura da kuskuren ba.
• Craming komai cikin hanzari guda ɗaya: Dogon mahallin yana da arha amma yana da tsada kuma ya rasa bayanan tsakiya.
• Samun makale a cikin tsararraki ba tare da aunawa ba: Idan amsar ba ta da kyau, fara tambayar "Shin sashin da ya dace ya zo?" ya kamata a tambayi.

A takaice

• RAG wata hanya ce da ke shigar da takardun da suka dace da tambaya a cikin samfurin a matsayin mahallin; baya canza ma'auni ("bude littafin jarrabawa").
• Kyakkyawan daidaitawa yana koyar da salo / tsari, RAG yana ba da bayanai na yanzu da takamaiman; dogon mahallin yana aiki da kyau don ƙananan ƙayyadaddun saiti. A mafi yawan al'amuran, RAG ana gwada shi da farko.
• Bututun yana da matakai guda biyu: indexing offline (chunk + embedding + save) da kuma tambayar kan layi (dawowa + da sauri + samarwa).
• RAG yana ba da lokaci, takamaiman bayani, tabbatarwa, sarrafa hallucination, da ƙarancin farashi.
• Ingancin tsarin kai tsaye ya dogara da ingancin maidowa: yanki mara kyau yana nufin amsa mara kyau.

Aikin aikace-aikace

Zaɓi tushen bayanai na gaske daga ƙungiyar ku (misali takaddar tsari ko shafin FAQ). (1) Rubuta tambayoyi na gaskiya guda 5 game da wannan tushen. (2) Lura da wane ɓangaren takaddar ya ƙunshi daidaitacciyar amsa ga kowace tambaya - wannan shine jerin "amsar zinare" naku. (3) Yin amfani da samfurin "ƙarfi mai ƙarfi" a sama, manna sashin da ya dace da hannu azaman mahallin kuma tambayi samfurin. (4) Kwatanta amsar da samfurin ya bayar tare da amsar zinare da alama a matsayin gaskiya/ƙarya. Wannan shine sigar farko ta hannun hannu na kima wanda zaku sarrafa ta atomatik a cikin raka'a masu zuwa.

jerin abubuwan dubawa

• [ ] Zan iya bayyana a cikin jumla ɗaya cewa RAG baya canza ma'auni, yana ƙara mahallin.
• [ ] Zan iya bambanta tsakanin RAG, daidaitawa mai kyau da dogon mahallin kuma lokacin da ya dace.
• [ ] Zan iya kirga fihirisa (tattara-shred-embed-ajiye) da kuma tambaya (samu-damar-sarrafa-sake) matakan tsari.
• [ ] Na san dalilin da ya sa na ƙara "idan ba a cikin mahallin ba, a ce ban sani ba" da kuma "bayyana tushen" umarni zuwa ga hanzari.
• [ ] Zan iya daidaita ka'idar "Kyauta Maidowa = ingancin RAG" zuwa nawa harka.