Riba:
- Aiwatar da nau'ikan bincike wanda ya haɗu da zaɓi na top-k da bincike na ma'ana da mahimmin kalmomi
- Ƙara daidaiton binciken farko tare da sake yin matsayi
- Samar da tambayoyi masu wahala da dabaru kamar canjin tambaya da HyDE
Kun share takaddun da kyau kuma kun sanya su cikin ma'ajin bayanai na vector. Yanzu ainihin aikin: ɗauko ɓangarorin da suka dace lokacin da mai amfani ya yi tambaya. Wannan shi ake kira maidowa kuma shine kashin bayan ingancin RAG. Tuna jumlar "Kyauta Maidowa = ingancin RAG"; Wannan rukunin shine ainihin fasahar haɓaka wannan ingancin. Za mu rufe dabaru masu amfani daga zaɓi na sama-k zuwa binciken gauraye, daga sake yin matsayi zuwa canjin tambaya.
Top-k: Nawa Za Mu Kawo?
Mafi mahimmancin saiti: guda nawa (k) don dawowa daga bincike. Idan ka kawo ƙasa da (k=1) akwai babban haɗarin rasa madaidaicin yanki; Idan kun ƙara da yawa (k = 20), zai ƙara ƙara zuwa samfurin kuma farashin alamar zai ƙaru.
Bambance tsakanin ra'ayoyi biyu. Tuna: ƙimar da daidaitaccen yanki yake cikin waɗanda aka dawo dasu. Daidaitawa: adadin abin da aka dawo da shi ya dace. Ƙara k yana ƙara tunawa amma yana rage daidaito. Manufar ita ce daidaita biyun.
saman-k
Tasiri
yanayin da ya dace
1-3
Babban madaidaici, haɗarin hasara
Tambayoyi masu sauƙi, amsa ɗaya
4-8
Ma'auni; mafi yawan al'amuran
Babban darajar RAG
10-20
Tunawa da yawa, hayaniya yana ƙaruwa
Tare da sake daraja (a ƙasa)
Tukwici: Tsarin gama gari kuma mai ƙarfi: ɗabo fadi (k=20), sannan kunkuntar tare da sake yin matsayi (saman 4). Ta wannan hanyar ba za ku rasa komai ba kuma kuna ba da mahallin tsabta ga ƙirar.
Binciken Haɓaka: Ƙarfin Bincike Biyu
Binciken Semantic (vector) yana ɗaukar ma'anar amma ya rasa abubuwa: cikakken lambar samfur ("XR-4471"), taƙaitaccen taƙaitaccen bayani, sunan da ya dace, lambar sigar. Don waɗannan ɗawainiya daidai-daidai, bincika kalmomin tsohuwar makaranta-musamman na al'adar algorithm da ake kira BM25-yana da kyau sosai.
Binciken da aka haɗa ya haɗa biyun: duka biyun bincike na ma'ana da mahimmin kalmomi suna aiki, haɗa sakamakon. Don haka, a cikin tambaya kamar "Lokacin Garanti na
Ana yin haɗe da yawa tare da RRF (Madaidaicin Rank Fusion): waƙar da ta fi girma a cikin jerin duka tana zuwa gaba.
# Maido da Haɓaka (conceptual)sem = vector_search(tambaya, k=20) # ma'ana key = bm25_search(tambaya, k=20) # fullwordresult = rrf_merge(sem, key)[:8] # fuse biyu, saman 8
Sake-daraja: Na biyu da Wayar Waya
Kiran farko na iya zama mai sauri amma rashin kunya. Sake saka maki ƴan takarar sun dawo da bincike na farko ɗaya bayan ɗaya tare da mafi ƙaƙƙarfan ƙira (yawanci mai rikodin giciye-samfurin da ke karanta tambaya da sashe tare kuma ya ƙididdige mahimmanci) kuma yana motsa waɗanda suka fi dacewa zuwa sama.
Ma'anar ita ce: bincike na farko yana ba da adadi mai yawa na 'yan takara don tunawa ('yan takara 20), mai sake sake zabar mafi kyawun 4 don daidaito. Wannan matakin mataki biyu ya fi daidai fiye da binciken mataki guda. Yana kashe ɗan jinkiri da ƙarin aiki; Riba shine babban haɓakar inganci.
# Maidowa mataki biyu (ra'ayi)' yan takara = hybrid_search(tambaya, k=20) # m, quickscore = reranker.score(tambaya, 'yan takara) # maki mai dacewa ga kowane ɗan takara mahallin = rated.rank ()[:4] # saman 4
Canza Tambaya
Wani lokaci matsalar ba a cikin bincike ba ne, amma a cikin tambayar kanta. Idan mai amfani ya tambaya "me game da ma'aikatan nesa?" ', wannan ba za a iya neman shi kadai ba (ba a bayyana abin da ke ga ma'aikatan nesa ba). Anan ga dabarun canza tambaya:
- Sake rubutawa: Yi amfani da tarihin tattaunawa don sanya tambayar ta tsaya kawai: "Mene ne ma'aikatan nesa suka cancanci izinin shekara?"
- Multi-questery: Ƙirƙirar maganganu daban-daban guda 3 na tambaya iri ɗaya, bincika tare da su duka, haɗa sakamakon. Kalmomi daban-daban suna samun guda daban-daban.
- HyDE (Hanyoyin Hasashen Tattalin Arziki): Da farko buga “amsar da za ta yiwu” ga ƙirar, sannan a haɗa kuma bincika wannan tambarin amsar. Amsar hasashe wani lokaci ana samun mafi kyau saboda ta fi kusa da kalmomi zuwa ainihin takaddar.
# Multi-query (conceptual)variants = model.uret("Bayyana wannan tambaya ta hanyoyi 3 daban-daban:" + question)tum_sonuc = [] don v a cikin bambance-bambancen: all_sonuc += vector_intermediate(v, k=6)context = mufuradi_da_oda(sakamakon sakamako)[:6]
Mai da Rauni / Ƙarfin Maidowa
Rarrauna (mataki ɗaya, ilimin tauhidi kawai, akai k=3):
sakamako = vector_search(tambaya, k=3)# Matsala: lambar samfur ta ɓace, ba za a iya shigar da daidai sashi na 3 a cikin tambayoyi masu wahala ba.
Ƙarfi (matasan + widek + sake tsarawa + tambaya da yawa idan ya cancanta):
'yan takara = hybrid_search (sake rubuta (tambaya, baya), k=20) mahallin = reranker.score (tambaya, 'yan takara) .farko (4) # Dukansu daidai daidai da ma'ana an kama su; Yana zuwa mafi kyawun samfuran 4.
Mini Cases guda uku
Case 1 - Lambar samfur ta tsere. Binciken ma'anar ma'ana mai tsabta akan ƙungiyar tallafi ba zai iya samun "RTX-9080" a zahiri ba a cikin tambayar "kuskuren direba na RTX-9080"; Yana kawo wasu kayayyaki masu irin wannan suna. Lokacin da aka ƙara binciken matasan, daidaitaccen lambar ya faru kuma adadin maido da ingantattun labarai ya ƙaru daga 58% zuwa 92%.
Case na 2 - Bambanci na sake fasalin. Dan-dan-agajin yana aiki tare da k=5 amma abin da ya dace shine 7-10 mafi yawan lokaci. Ya kasance a cikin sahu. Lokacin da aka gabatar da k=20 + sake yin matsayi, ƙimar madaidaicin labarin da ke cikin saman 3 ya ƙaru daga 61% zuwa 94%; Latency ya ƙaru da 300ms kawai - sulhu mai karɓuwa.
Shari'a 3 - Tambaya ta biyo baya ba tare da mahallin ba. A cikin mataimaki na HR, mai amfani yana tambaya "Me game da masu aikin ɗan lokaci?" Lokacin da na tambaya, tsarin ya kawo sassan da ba su dace ba. Ta hanyar sake rubuta tambayar (ciro mahallin "halin shekara-shekara" daga baya da kuma ƙara "ma'aikatan lokaci-lokaci suna da damar izinin hutu na shekara"), ƙimar amsa daidai ta karu daga 40% zuwa 86%.
Kuskuren gama gari
- Dogaro kawai akan binciken ilimin ma'ana: An rasa lambar samfur, gajarta, sunan da ya dace; Ƙara matasan.
- Tsayawa k yayi ƙanƙanta: Madaidaicin yanki ba zai iya shigar da lissafin ba; yi shi fadi da kunkuntar shi tare da sake daraja.
- Kada ku taɓa tunanin sake yin matsayi: Binciken farko rashin mutunci ne; Na biyu mai wayo wucewa yana inganta inganci sosai.
- Neman tambayoyi masu biyo baya kamar haka: Tambayar da ba ta da ma'ana tana neman maras ma'ana; sake rubutawa.
- Makafi yana ƙaruwa k (ba tare da sake daraja ba): Samfurin yana cike da hayaniya, amsawar ta ɓace kuma farashin yana ƙaruwa.
Hattara: Kowace dabara tana ƙara farashi da jinkiri. Tambaya da yawa na nufin bincike sau 3, sake-daraja yana nufin ƙarin kiran ƙira. Fara da sauƙi matasan + m k farko; Auna kuma ƙara dabaru masu nauyi a inda ake buƙata da gaske. Ƙara ba tare da aunawa ba.
A takaice
- Top-k shine ma'auni tsakanin tunawa da daidaito; Gabaɗaya 4-8 farawa ne mai kyau.
- Binciken Haɓaka yana haɗa bincike na ma'ana tare da bincike mai mahimmanci (BM25); Yana dawo da ainihin matches.
- Sake sakawa yana sake saka ƴan takara daga faffadan bincike na farko tare da ƙaƙƙarfan ƙira kuma ya zaɓi mafi kyawu.
- Canjin tambaya (sake rubutawa, tambaya da yawa, HyDE) yana sa tambayoyin da ba su da wahala kuma babu mahallin bincike.
- Ƙaƙƙarfan tsari: faɗo mai faɗi → haɗuwa tare da matasan → kunkuntar tare da sake matsayi; amma ƙara kowace dabara ta hanyar auna ta.
Aikin aikace-aikace
Shirya tambayoyi masu wuyar gaske 6 tare da bayanai daga raka'o'in da suka gabata: aƙalla 2 suna buƙatar daidaitattun matches (lambar samfur, taƙaitaccen lokaci, kwanan wata), ma'anar 2 (maudu'i iri ɗaya tare da kalmomi daban-daban), tambayoyin biyo baya 2 ba tare da mahallin ba. (1) Da farko ka yi la'akari da kowace tambaya a matsayin bincike na ma'ana mai tsarki kuma da hannu da hannu waɗanne sassa za su fito. (2) Sake kimanta tambayoyin iri ɗaya tare da haɓaka da ƙara ƙarin matsayi. (3) Sake rubuta tambayoyin biyo baya ba tare da mahallin mahallin ba. Yi la'akari a cikin tsari na tebur wanda fasaha ke yin bambanci a cikin nau'in tambaya.
jerin abubuwan dubawa
- [ ] Na san top-k shine ma'auni na tunowa da madaidaicin farawa.
- [ ] Zan iya bayyana dalilin da yasa binciken matasan ke dawo da ainihin matches.
- [ ] Zan iya amfani da dabaru na mataki biyu na sake yin matsayi (fadi → kunkuntar mai wayo).
- [ ] Na gane buƙatar sake rubuta tambayoyin biyo baya ba tare da mahallin ba.
- [ ] Na tabbatar da cewa na kara nauyi dabaru ta hanyar aunawa, ba ta hanyar aunawa ba.