Riba:
- Ability don zaɓar ma'aunin da ya dace da nau'in matsala da mahallin kasuwanci (PR-AUC / tuna a cikin bayanan da ba daidai ba, MAE / RMSE a cikin koma baya) da gane kan / ƙarƙashin koyo
- Ikon fassara kowane ma'auni a kan tushen tushe da auna karkacewa da raba hayaniya daga ci gaba tare da tabbatarwa
- Ikon bayar da rahoton sakamako mara kyau ta hanyar daidaita hyperparameters tare da saitin tabbatarwa da amfani da saitin gwajin kawai a ƙarshen.
Horon samfurin (horarwa: tsarin koyo daga bayanai) shine mafi bayyane amma mafi kuskuren mataki na injiniyan ML. Ya bayyana saboda yana samar da adadi mai gamsarwa kamar "daidai 95%". Batar da kai domin wannan lambar galibi ita ce amsar da ta dace ga tambayar da ba ta dace ba. A cikin wannan rukunin, ana tattauna ilimi da kimantawa tare da horon injiniya; Muna amfani da basirar wucin gadi a matsayin abokin tarayya a cikin ƙirar gwaji da kuma kafa yanke shawara akan aunawa.
Hankali na sake zagayowar horo
Samfurin yana koyon tsari a cikin bayanai ta hanyar rage girman aikin asara: aikin da ke auna kuskuren ƙira ta lambobi. Algorithm na ingantawa (misali zuriyar gradient) yana rage asara ta hanyar daidaita sigogi mataki-mataki. Manufar ba don haddace bayanan horo ba, amma don haɗa su zuwa bayanan da ba a taɓa gani ba.
Manyan hatsarori guda biyu:
- Overfitting: Samfurin yana haddace bayanan horo kuma ya kasa kan sabbin bayanai. Nasarar horarwa tana da girma, nasarar tabbatarwa ba ta da yawa.
- Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙarƙashin Ƙirar Ƙirar ba zai iya ɗaukar tsarin ba; Dukansu horo da nasarar tabbatarwa suna da ƙasa.
Neman ma'auni shine fasaha na ilimi. Saitin tabbatarwa yana nan don saka idanu akan wannan ma'auni: idan nasarar tabbatarwa ta fara raguwa yayin da nasarar horarwa ta karu, an fara koyo.
Tukwici: Yi ƙirƙira asarar horo da tabbatarwa tare a kowane mataki. Batun da ƙugiyoyin biyu suka fara rarrabuwa shine inda ake fara koyo kuma shine lokacin da ya dace don "tsayawa da wuri".
Zaɓin awo: yanke shawara mafi mahimmanci
Ma'aunin da ba daidai ba yana sa samfurin mai kyau yayi kyau kuma mummunan samfurin yayi kyau. Matsalar tana ƙayyade ma'auni:
- Rarraba mara daidaituwa (aji ɗaya ba kasafai ba ne, misali zamba): Daidaito yaudara ne. Samfurin da ya ce "kira komai na al'ada" zai sami daidaiton kashi 99% amma ba zai kama zamba ɗaya ba. Madadin haka, daidaito (nawa ne abin da na kama yana da inganci), tuna (nawa ne abin da na kama gaskiya ne) kuma ana amfani da ma'aunin su F1 ko PR-AUC.
- Daidaitaccen Rarraba: Daidaitawa da ROC-AUC na iya dacewa.
- Regression (ƙididdigar lamba): MAE (ma'anar kuskure cikakke), RMSE (yana hukunta manyan kurakurai), MAPE (kuskuren kashi).
- Matsayi/shawarwari: NDCG, MRR, Tuna @K.
Ko daidaici ko tunawa yana da mahimmanci ya dogara da mahallin kasuwanci. Tuna (ba a rasa kowane mara lafiya) shine fifiko a cikin gwajin cutar kansa; Daidaitacce a cikin tace spam (ba aika mahimman imel zuwa spam ba) yana da mahimmanci. Wannan shawara ce ta kasuwanci, ba ta fasaha ba, kuma injiniya da mai shi ne suka yi tare.
Rauni mai ƙarfi / Ƙarfi mai ƙarfi
Rauni mara ƙarfi: "Kimanta aikin ƙirara, daidaito 0.97."
Ƙarfi mai ƙarfi: "Ina da samfurin gano zamba; ƙimar aji mai kyau shine 1.5%. Ana ba da rahoton daidaito kamar 0.97. Bayyana dalilin da yasa wannan ma'auni na iya zama mai ɓatarwa, gaya mani wane ma'auni (daidaici, tunawa, PR-AUC) Ya kamata in fi so da kuma me yasa. Har ila yau ƙididdige yadda daidaitaccen 'kira duk abin da ba daidai ba' samfurin asali zai iya ganin ainihin wannan bayanin, don haka ƙara da darajar.
Bambance-bambance: hanzari mai ƙarfi yana ba da rabon aji da mahallin kasuwanci; yana kuma neman kwatancen ƙirar ƙira - wannan shine mafi mahimmancin anka don ko awo yana da ma'ana.
Baseline: awo ba tare da kwatanta ba bashi da ma'ana
Ma'auni ba shi da kyau ko mara kyau da kansa; Yana da kyau ko mara kyau bisa ga samfurin asali. Samfurin asali shine mafita mafi sauƙi wanda ke zuwa hankali: "koyaushe gaya wa yawancin aji", "maimaita ƙimar makon da ya gabata", "kimanin ma'anar". Idan samfurin ku ba zai iya wuce wannan sauƙi a fili ba, duk rikitarwa ba don komai ba ne.
Tsanaki: Jumlar "Tsarin nawa shine 85% daidai" ita kanta bata ce komai ba. Idan samfurin tushe ya riga ya sami 84%, samfurin ku kusan ba shi da amfani; Idan samfurin tushe ya sami 50%, samfurin ku cikakke ne. Koyaushe magana cikin sharuddan asali samfurin.
Tsare-tsare da aminci
Rarraba horo/gwaji ɗaya na iya zama saboda dama. Ƙaddamarwa ta giciye: rarraba bayanai zuwa sassan k da gwada kowane sashi a jere yana nuna yadda aikin ya kasance. A cikin ingancin giciye sau 5 za ku sami maki daban-daban guda biyar; Ma'anarsu da daidaitattun karkatattun su suna da mahimmanci. Idan matsakaita shine 80% amma karkacewar shine ± 12%, ƙirar ku ba ta da ƙarfi - yana iya bambanta sosai a cikin bayanan na gaba.
Wannan kuma yana da mahimmanci ga "kwatancen samfuri biyu". Idan Model A ya sami 81% kuma Model B ya sami 82%, shin B ya fi kyau? Idan karkacewar shine ± 3%, wannan bambanci na iya zama amo. Yi la'akari da ko bambancin yana da mahimmanci kafin yanke shawara.
Gyaran hyperparameter: tare da inganci, ba gwaji ba
Hyperparameters (saituna da aka ƙaddara da hannu kafin horo - ƙimar koyo, zurfin bishiya, da sauransu) an saita tare da saitin tabbatarwa. Ana amfani da saitin gwajin kawai a ƙarshe, sau ɗaya. Idan ka zaɓi hyperparameters ta hanyar duba saitin gwajin, saitin gwajin zai zama gurɓata kuma aikin da kuka ba da rahoton zai kasance da kyakkyawan fata wanda ba haka bane a zahiri.
Hankali na wucin gadi shine mataimaki mai kyau wajen tsara sararin binciken hyperparameter da rubuta lambar bincike (binciken grid, binciken bazuwar, haɓakar Bayesian). Amma har yanzu kuna yanke shawarar "wane awo za mu inganta?"
uku mini lokuta
Case 1 - Daidaitaccen tarko. Ƙungiyar likitoci ta yi alfahari da samfurin da ya gano wata cuta mai wuya: 98% daidaito. Lokacin da aka yi kwatancen samfurin asali, gaskiyar ta bayyana: tun da cutar ta kasance 2%, ƙirar da ta ce "kira kowa lafiya" shima ya sami 98%. Tunawa da samfurin ya kasance 11% kawai - bacewar yawancin marasa lafiya. Da zarar an canza awo zuwa PR-AUC, an auna ainihin aikin kuma an sake fasalin ƙirar.
Hali na 2 - Kuskure amo don ci gaba. Tawagar ta shafe watanni tana haɓaka samfurin daga 86.2% zuwa 86.9%. Tabbatar da giciye ya nuna cewa son zuciya shine ± 1.4% - don haka ma'anar 0.7 "ingantawa" shine hayaniya na ƙididdiga. Tawagar ta yi hasarar makwanni uku kan kudaden da ba na gaskiya ba. Darasi: kar a bayyana nasara ba tare da tabbatar da cewa ingantaccen ya fi karkata ba.
Case na 3 - Lalacewar saitin gwajin. Wani injiniya ya sake duba saitin gwajin don zaɓar mafi kyawun hyperparameters. Kashi 91% da aka ruwaito ya ragu zuwa 83% a samarwa. Me ya sa: ya zaɓi samfurin bisa ga rashin sani ta hanyar kallon gwajin da aka saita akai-akai (koyo kan tsarin gwajin). An ba da rahoto da aikin ainihin aiki tare lokacin da aka yi amfani da saitin tabbatarwa daban.
Samfura masu kwafi
Taimaka min zaɓar ma'aunin da ya dace don wannan matsalar rarrabuwa. Matsala: [abin da aka annabta] Rarraba aji: [tabbatacciyar ƙimar
Yi la'akari da kwatankwacin waɗannan samfura biyu masu zuwa. Model A Tabbatar da giciye: [jerin ƙididdiga]Model B giciye-tabbatacce: [jerin maki] Yi ƙididdige ma'ana da daidaitaccen karkacewa. Shin bambancin yana da mahimmancin ƙididdiga ko kuwa hayaniya ce kawai a cikin karkacewar? Wanne zaku ba da shawarar in zaɓi kuma me yasa?
Duba wannan lambar horarwa don masu zuwa:1) Shin an zaɓi hyperparameters tare da saitin gwaji ko saitin tabbatarwa?2) Ana kula da tsayawa da wuri tare da daidaitaccen saiti?3) Shin akwai alamun haɓakar koyo (bambancin asarar horo / tabbatarwa)? Code: [lambar]
Wanne daga cikin MAE, RMSE, da MAPE zan bayar da rahoton wannan matsala ta koma baya? Sikelin madaidaicin manufa: [kewaye] Shin manyan kurakurai ba su da kyau (RMSE), ko kuma duk daidai suke (MAE)?
Teburin zaɓi na awo
Nau'in matsala
Daidaitaccen awo
Don kaucewa
Me yasa
Rarraba mara daidaituwa
PR-AUC, F1, tuna
Daidaito
Yawancin aji yana haɓaka awo
Madaidaicin rarrabawa
Daidaitacce, ROC-AUC
-
Dogara a daidaitattun bayanai
Regression (muhimmi mai mahimmanci)
RMSE
MAPE (idan sifili)
Yana hukunta manyan kurakurai
Juya baya (daidai nauyi)
MAE
-
Sauƙin fassara
Matsayi/shawarwari
NDCG, Tuna @K
Daidaito
Oda yana da mahimmanci
Kuskuren gama gari
- Yin amfani da daidaito akan bayanai marasa daidaituwa. Mafi yawan kuskuren awo.
- Ba yin kwatancen samfurin tushe ba. Yana sa ma'aunin ya rasa ma'anarsa.
- Duban saitin gwajin don hyperparameters. Kyakkyawan fata, sakamako mara gaskiya.
- Dogaro da ɗaki guda ɗaya. Idan ba tare da tabbatarwa ba ba za ku ga son zuciya ba.
- Kuskure amo don ci gaba. Ƙananan "ingantawa" daga karkacewa galibi sa'a ne.
- Yin watsi da mahallin kasuwanci. Daidaitaccen ma'auni / tunawa shine yanke shawara na kasuwanci.
A takaice
Haƙiƙanin fasaha a cikin horarwar ƙirar ba don samar da adadi mai yawa ba, amma don sanin abin da lambar ke nufi. Matsala da mahallin kasuwanci sun ƙayyade ma'auni daidai; fassara kowane awo bisa ga tsarin asali; auna son zuciya tare da tabbatarwa kuma kada ku kuskure hayaniya don ci gaba; Yi amfani da saitin gwajin kawai a ƙarshe, lokaci ɗaya. AI shine abokin tarayya a cikin ƙirar gwaji, amma shawarar "mai kyau" ya rage na ku da naku.
Aikin aikace-aikace
Don samfurin rarrabuwa: (1) rubuta rabon aji, (2) gina ƙirar tushe mai dacewa kuma auna makinsa, (3) kimanta ƙirar ku tare da ingancin giciye mai ninki 5 kuma bayar da rahoton ma'ana da ma'auni, (4) ƙididdige ma'aunin da ya dace da matsalar (misali PR-AUC) maimakon daidaito. Rubuta ko samfurin ku ya doke ƙirar tushe da babban gefe ba tare da son zuciya ba.
jerin abubuwan dubawa
- [ ] Na zaɓi awo bisa ga nau'in matsala da mahallin kasuwanci.
- [ ] Na gina samfurin tushe kuma na kwatanta da shi.
- [ ] Na bayar da rahoton ma'anar da karkacewa tare da tabbatarwa.
- [ ] Na tabbatar da cewa ingantawa ya fi karkata.
- [ ] Na zaɓi hyperparameters tare da saitin tabbatarwa.
- [ ] Na yi amfani da saitin gwajin sau ɗaya kawai, a ƙarshe.