Gabatarwa zuwa Hankali na Artificial a Kimiyyar Bayanai da Nazarin: Gudun Aiki, Matsayi, Iyakoki, Tabbatarwa da Da'a

Riba:

• Ability don rarrabe matakan aiki shida na kimiyyar bayanai (ma'anar matsala, tattarawa, tsaftacewa, ganowa, ƙirar ƙira, sadarwa) da kuma ikon da ke aiki da hankali na wucin gadi a kowane mataki, bisa ga matakin haɗarin aiki.
• Ƙarfin yin amfani da horo wanda ke tabbatar da kowane fitowar hankali na wucin gadi ta hanyar haɗa shi zuwa tushen, gudu da kwatanta shi, da kuma wucewa ta hanyar tacewa ƙwararru.
• Ikon juyar da haɓakawa da ka'idodin sirri (rubutu cikin lamba, ɓoyewa) zuwa halaye na bincike daga rana ɗaya.

Raw, m, kuma galibi “karya” bayanai suna sauka akan teburin masanin kimiyyar bayanai kowace rana. A cikin teburin tallace-tallace, an rubuta ginshiƙin kwanan wata a cikin nau'i uku daban-daban; Lambobin abokan ciniki ba su da komai a wasu layukan; Sunan shafi shine "shigarwa", amma yana ƙunshe da ƙimar TL da USD. Aikin kimiyyar bayanai shine yanke shawara mai dogaro daga cikin wannan hargitsi: tattara bayanan, tsaftace su, bincika su, gina samfuri, tabbatar da sakamakon, kuma juya shi cikin labari. Hankali na wucin gadi (AI, ko AI a takaice-tsarin kwamfuta wanda zai iya samar da rubutu da lamba, gane alamu, taƙaitawa, da yin tsinkaya) na iya taɓa kowane hanyar haɗi a cikin wannan sarkar: rubuta lambar tsaftacewa a cikin mintuna, ba da shawarar zane-zane don binciken bincike, fassara ma'aunin ƙirar ƙira, gyara tambayar SQL. Amma AI guda ɗaya kuma na iya ba ku ƙirƙira ƙirƙira kamar "daidaitawa 0.72" don bayanan da bai taɓa gani ba.

Wannan rukunin farko ba gabatarwar laburare bane. Manufarsa ita ce bayyana inda za a saka AI a cikin aikin ilimin kimiyyar bayanai da kuma inda ba za a saka shi ba. Bari mu tsara ƙa'idar asali tun daga farko: AI mataimaki ne, ba masanin kimiyyar bayanai ba. Shawarar irin bayanan da za a tattara, wane awo da za a yanke shawara a kai, ko za a saka samfuri a cikin samarwa, da kuma inda za a zana iyakoki na ɗa'a ya rataya kan ƙwararren ƙwararren. Fitowar AI da ba a tantance ba yana da haɗari, kamar rahoton bincike da ba a sanya hannu ba.

Gudun aikin kimiyyar bayanai: taswirar ƙarshen-zuwa-ƙarshe

Don fahimtar aikin bayanai, yana da taimako a rarraba shi zuwa matakai shida. Wannan tsarin duka yana bin wannan taswira.

1. Ma'anar Matsala: Menene muke auna, me yasa, don tallafawa wace shawara? Ba a wakilta wannan lokaci ga AI; Mutum ne ke ayyana aikin.

2. Tarin bayanai: Gano tushe, fitar da bayanai, samfuri. (Raka'a 2)

3. Data tsaftacewa da preprocessing: Rasa darajar, outlier, irin tuba. (Raka'a 3)

4. Binciken bayanan bincike (EDA - Binciken Bayanan Bincike, mataki na gane rarrabawa da alaƙar bayanan "ta ido"): (Saka ta 4)

. (Kashi na 5, 6, 7)

6. Sadarwa da samarwa: Kayayyakin gani, labari, MLOps (ladabi na ɗaukar samfurin rayuwa da saka idanu). (Kashi na 8, 11)

AI yana aiki tare da iko daban-daban a kowane ɗayan waɗannan matakai shida. Teburin da ke ƙasa ya taƙaita wannan rabon iko; Wannan shine tebur mafi mahimmanci guda ɗaya na module.

Mataki

Matsayin AI

Matsayin haɗari

Wanda ya yarda

Ma'anar matsala

kwakwalwar abokin tarayya

ƙananan

Masanin kimiyyar bayanai + masu ruwa da tsaki

Rubuta lambar tsaftacewa

Code sketch janareta

matsakaici

Masanin kimiyyar bayanai (karanta lambar)

Bayanin EDA

mai ba da shawara

matsakaici

masanin kimiyyar bayanai

Ƙirƙirar fasali

Idea da code janareta

matsakaici-high

Sarrafa yatsa ya zama dole

Zaɓin samfurin / awo

mai ba da shawara

babba

masanin kimiyyar bayanai

Shawarar da za a yi a samarwa

shigarwar taimako

mai girma

Ƙungiya + mai kasuwanci

Amincewa da ɗa'a/keɓancewa

mai kara kuzari

mai girma

mutum, kullum

Yi la'akari da jumla ɗaya daga wannan ginshiƙi: yayin da hada-hadar ta tashi, aikin AI yana raguwa kuma amincewar ɗan adam yana girma.

Me yasa tabbatarwa shine zuciyar wannan kasuwancin

Samfuran Harshen AI sun tabbata, amma ƙila ba za su tabbata ba. A cikin harshe na fasaha, ana kiran wannan hallucination: shine ƙirar ƙirƙira na bayanan da ba su wanzu a cikin jimla mai kyau kamar gaskiya ne. A kimiyyar bayanai, wannan ya zo ta hanyoyi uku. Na farko lambar karya ce: idan ka tambayi samfurin "menene matsakaicin adadin oda" ba tare da ba da wani bayanai ba, zai gaya maka lamba da gaba gaɗi, kodayake bai taɓa ganin bayananka ba. Na biyu shine aikin da ba ya wanzu: ƙirar na iya ba da shawarar aikin da babu shi kwata-kwata, kamar pandas.read_excel_fast(). Na uku, fassarar ƙididdiga ba daidai ba: ƙirar na iya maimaita kuskuren ƙididdiga na yau da kullun kamar "p-darajar shine 0.04, don haka hasashen shine 96% daidai".

Horon tabbatarwa ya ƙunshi matakai guda uku:

1. Hanyar haɗi zuwa tushe: Kowane lamba, ƙima, da yanayin ya kamata su fito daga bayanan ku, ba ƙwaƙwalwar AI ba. Yi amfani da AI don lambar da ke aiki akan bayanai, ba don ta tuna da bayanai ba.
2. Gudu da kwatanta: Gudanar da kowane yanki na lambar da AI da kanku ya ba ku; Kwatanta kowane awo da yake samarwa zuwa tushe mai zaman kansa.
3. Tace gwani: Gwada wa kanku ko fitarwa ya saba wa kididdiga da ilimin yanki.

Tsanaki: Sanya binciken da AI ta rubuta a cikin gabatarwa ba tare da gudu ba da karanta shi kamar gabatar da rahoton da ba a sanya hannu ba. Don kawai lambar tana aiki ba yana nufin daidai ba ne; Lambar da ke taƙaita ginshiƙi mara kyau kuma yana aiki ba tare da kurakurai ba.

Maimaituwa: iya samar da sakamako iri ɗaya sau biyu

Shiru amma mahimmancin ƙa'idar kimiyyar bayanai shine sake fasalin: lokacin da kuka sake gudanar da bincike bayan watanni shida ko akan wata kwamfuta daban, kuna samun sakamako iri ɗaya. Wannan haɗarin yana ƙaruwa lokacin aiki tare da AI, saboda lokacin da kuka gaya wa AI don "yin wannan jadawali" kuma ku kunna shi da hannu, matakan sun ɓace. Doka: kowane mataki dole ne a yi rajista a cikin lambar da sarrafa sigar (kamar Git); A cikin matakan da suka shafi bazuwar, nau'in bazuwar (ƙimar farko na janareta na lambar bazuwar; idan an kayyade, sakamakon zai zama mai maimaitawa) yakamata a gyara shi. Za mu zurfafa wannan batu a cikin Raka'a 10; A yanzu, shiga cikin wannan al'ada: "Kada ku danna da hannu, rubuta cikin lamba."

uku mini lokuta

Case 1 - Amintaccen hanzari. Wani manazarcin kasuwancin e-commerce yawanci zai shafe rabin yini yana share teburin oda na layuka 240. Ya ba da sunayen ginshiƙai da layuka 5 na farko (ba tare da bayanan sirri ba) ga AI kuma ya nemi lambar tsabtace pandas. AI ya samar da daftarin aiki a cikin 8 seconds; Manazarcin ya karanta layin lambar ta layi, ya gyara kuskure a cikin tantancewar kwanan wata, sannan ya gudanar da shi. Duration: 35 minutes maimakon 4 hours. AI ta ba da lambar, tabbaci ya kasance tare da ɗan adam.

Case 2 - The made up metric trap. Wani ma'aikaci ya tambayi AI, "Yaya daidai dajin dajin akan wannan bayanan?" ba tare da horar da samfurin kwata-kwata ba. "Kusan 89%," in ji AI. Kwararren ya sanya wannan a cikin gabatarwa; Lokacin da aka horar da ainihin samfurin, daidaito ya kasance 71%. Kuskure: Jiran awo ba tare da bada bayanai da samfura ga AI ba.

Hali na 3 - Silent yayyo. Ƙungiya ɗaya ta aiwatar da ra'ayin AI da aka ba da shawara na "ƙara ma'anar madaidaicin manufa" ba tare da tambayar shi ba. Samfurin ya yi kashi 98% akan saitin gwajin amma ya fado a samarwa saboda fasalin yana yoyo bayanan nan gaba (bayan bayanai, Unit 10). Kuskure: Ba ƙididdigewa tace shawarar AI ba.

Rauni mai ƙarfi / Ƙarfi mai ƙarfi

Rawanin faɗakarwa:

Yi nazarin bayanan tallace-tallace kuma ku gaya mani matsakaicin kudaden shiga.

Wannan da'awar ba ta da lahani: ba a ba da bayanan AI ba, don haka "matsakaicin kudin shiga" kawai za a iya samu. Babu ginshiƙai, ko lokacin, ko kuɗin da ba su bayyana ba.

Ƙarfi mai ƙarfi:

Matsayinku: mataimaki na taimakawa masanin kimiyyar bayanai. Ina da pandas DataFrame: df. Shafukan: - order_date (rubutu, a cikin tsari "2024-03-01") - adadin_tl (decimal, NaN a wasu layuka) - abokin ciniki_id (integer) Aiki: (1) rubuta lambar pandas wanda ke ƙididdige matsakaicin adadin, (2) bayyana yadda yake sarrafa ƙimar NaN, (3) jera zato na lambar. Kada ku haɗa abubuwan da ke cikin bayanan; kawai samar da code kuma zan gudu da kuma tabbatar da sakamakon.

A cikin wannan buƙatar, makirci, tsammanin da "haramcin ƙirƙira" sun bayyana a fili. Har yanzu kuna gudu kuma ku tabbatar da fitarwa da kanku.

Mafarin xa'a da sirri

Kimiyyar bayanai galibi tana aiki tare da bayanan sirri: abokin ciniki, haƙuri, bayanan ma'aikata. KVKK (Dokar Kariyar Bayanan Sirri) a Turkiye da GDPR a Turai suna kare wannan bayanan. Manna ainihin sunan ku, ID, imel ko adireshin ku cikin kayan aikin AI na jama'a cin zarafi ne. Doka: ɓoye bayanan kafin rabawa, samar da tsari kawai (sunayen ginshiƙi, nau'ikan) da layukan misali idan ya cancanta. Za mu zurfafa batun xa'a a cikin Sashe na 11; Bari mu sanya ka'ida daga farkon: Don fahimtar bayanan, raba tsarinsa, ba abun ciki ba, sau da yawa isa.

Kuskuren gama gari

• Jiran lambobi / awo ba tare da bada bayanai ga AI ba. Samfurin ba zai iya samun damar bayanan ba; Lambar da ya bayar na bogi ne. Koyaushe a lissafta sakamakon kuma a yi aiki.
• Tunanin cewa lambar aiki daidai ne. Lambar da ke sarrafa ginshiƙi mara kyau kuma tana gudana ba tare da kurakurai ba; Kada ku dogara ba tare da karanta dabaru ba.
• Manna bayanan sirri na ainihi a cikin buɗaɗɗen kayan aiki. Suna, lambar ID, imel ɗin ba a taɓa rabawa ba; Tsarin ya isa.
• Yin matakan da hannu kuma ba rubuta su cikin lambar ba. Binciken da ba a iya sakewa ba shi da tabbas.
• Yarda da fassarar AI na ƙididdiga ba tare da tambaya ba. AI na iya maimaita kurakurai na yau da kullun a cikin ra'ayoyi kamar p-darajar da daidaitawa.

Tukwici: Haɗa ɗan gajeren "layin doka" a cikin kowane zaman AI na ku: "Kada ku tsara abubuwan da ke cikin bayanan; kawai samar da lamba / nazari kuma zan gudu kuma in tabbatar da sakamakon; nuna 'ba tabbata' inda ba ku da tabbas." Wannan jumla guda ɗaya tana rage haɗarin hasashe sosai.

a takaice

AI mataimaki ne mai ƙarfi a cikin ilimin kimiyyar bayanai: yana haɓaka lambar tsaftacewa, fassarar EDA, shawarwarin ƙira da gani. Amma ma'anar matsala, zaɓin awo, shawarar samarwa da iyakokin ɗa'a na ɗan adam ne. Gudun aikin yana da matakai shida, kuma aikin AI ya zama ƙarami yayin da haɗarin ke ƙaruwa. Halaye guda uku sune ginshikin komai: tushen haɗin gwiwa (tabbatacce), reproducibility (coding), da ɓoyewa (tsari). Da zarar kun shigar da waɗannan ukun, kowane kayan aiki a cikin sauran tsarin ya zama amintaccen mai haɓakawa a gare ku.

Aikin aikace-aikace

Kawai yi tsari na ƙaramin tebur ɗin da kuke da shi (ko hasashe): sunaye na shafi, nau'ikan, da layukan misalan 2-3 (ba tare da ainihin bayanan sirri ba). Tambayi AI don taƙaitaccen lambar ƙididdiga ta amfani da samfurin "Ƙarfin faɗakarwa" a sama. Gudanar da lambar, kwatanta abin da aka fitar zuwa ƙimar hannun hannu, bincika kowane zato, kuma lura da bincikenku a cikin maki 5.

jerin abubuwan dubawa

• [] Shin kawai na ba AI tsari da samfurin karya maimakon ainihin bayanan sirri?
• [ ] Na karanta kuma na gudanar da lambar da ta samar a layi-layi?
• [ ] Shin na tantance kowace lamba da kanta a cikin abin da ake fitarwa?
• [ ] Na rubuta kowane mataki na bincike a cikin lambar kuma na sanya shi maimaituwa?
• [ ] Shin na ƙaddara matakin haɗarin manufa kuma na sanya yanke shawara ta ƙarshe ga ɗan adam?