Unit 1 / 11

Introduction to Artificial Intelligence in Cybersecurity: Roles, Boundaries, Defense Ethics and Verification

Gains:

  • Being able to distinguish where artificial intelligence saves time in the defensive security workflow (detection, analysis, intervention, improvement, reporting) and where security-critical decisions (attack declaration, isolation, blocking, official report) are left to the analyst, depending on the task risk level.
  • Ability to apply the discipline of connecting each AI output to raw evidence (log, IOC, CVE, code), independently checking it and passing it through context filtering
  • Ability to anonymize log and security data within the scope of KVKK/privacy and to get into the habit of using only authorized, defensive purposes and with written permission.

In a security operations center (SOC in English - Security Operations Center; the team that monitors the organization's network, servers and users 24/7), thousands of event records flow every second. An employee connected to a server in Russia at 3:14 a.m.: is this an attack or a business trip abroad? One user encrypted 4,000 files in five minutes: is this ransomware or a backup tool? An email says "Invoice attached": is this a real accounting email or phishing? In a code review, a SQL query directly concatenates user input: is this an exploitable vulnerability or a secure script running on the internal network? Many of these questions are repetitive and tiring; Some of them are decisions that can directly lead to a data breach, millions of liras of damage, or the reputation of an institution.

Artificial intelligence (AI, or AI for short—computer systems that can scan, summarize, classify, flag anomalies, and produce drafts of large amounts of text and patterns) fits right in the middle of this picture. When used correctly, it summarizes thousands of lines of logs in seconds, prioritizes a cluster of vulnerabilities, analyzes a phishing email in seconds instead of minutes, and gives you time to think. When used incorrectly, it can ignore a real attack by labeling it as "normal", falsely alarm the team by fabricating a threat that does not exist, or leak confidential log data outside the organization.

The purpose of this unit is not a vehicle promotion. The goal is to clarify where to put AI in a security professional's job and where not to put it at all. Let's reiterate the basic principle from the beginning: Artificial intelligence is an assistant, not a decision-making authority in place of the security analyst. It is up to the qualified expert to declare an incident a real attack, isolate a system, block a user, and turn a finding into an official report. An unverified AI output is an unproven claim. And the reddest line of this module: Everything explained here is for defensive (defensive) purposes. Using AI to infiltrate a system without permission, create an attack tool, or conduct unauthorized testing is both illegal and outside the scope of this module.

Security workflow and the place of AI

To understand the business of defensive security, it is useful to break the process into five stages. Detection: Capturing suspicious behavior from log and SIEM data. Analysis/triage: assessing and prioritizing whether an alarm is real or false (false positive). Response: containment of the event, isolation, cleaning. Remediation: Closing the vulnerability, eliminating the root cause. Reporting: translation of the finding into technical and managerial documentation. AI can touch all five stages, but not each with the same authority.

Let's define a few terms from the beginning. SIEM (Security Information and Event Management) is a system that collects and correlates log records from different sources (server, firewall, application) and generates rule-based alarms. A false positive is when an event that is not actually a threat produces an alarm; It is a pain in the ass that tires out SOC teams and leads to “alert fatigue.” A false negative is when a real attack is never caught; It is the most dangerous mistake because it causes harm silently. IOC (Indicator of Compromise) is the technical trace that shows the trace of an attack: a malicious IP address, a file hash (hash), a domain name. TTP (Tactics, Techniques, Procedures) is a behavioral pattern that describes how the attacker behaves.

The following table summarizes the role and risk level of AI by mission:

Quest

Role of AI

Risk level

Who approves

Log summarization, noise reduction

accelerator, summator

low

analyst

Vulnerability prioritization outline

Sorter, suggestion

Low-Medium

analyst

Phishing email analysis

Pre-qualifying, clarifying

medium

analyst

Alarm triage (true/false)

Suggestion produces justification

Medium-High

Analyst (still correct)

Incident response playbook draft

sketch generator

Medium-High

Senior analyst / IR leader

Secure code review finding

Second eye, pointer

Medium-High

Developer + security

System isolation / blocking decision

not helpful

very high

authorized analyst

Official incident report/notification

Draft, expert corrects

very high

IR leader + legal/compliance

Keep in mind the one line in this chart: as risk rises, AI's role shrinks, human approval grows. No line of AI can exempt an event from review.

Why verification is the heart of this business

Artificial intelligence seems confident in the output it gives, but it may not be sure. A language model may fabricate a non-existent CVE number (vulnerability ID), refer to a log line that does not actually exist, or claim that an IP address is "malicious" without any evidence; this is called hallucination. The same model may also miss a real attack chain. Both traps come with equal fluidity; The only thing that separates right from wrong is your expertise and your habit of verifying.

The verification discipline consists of three steps:

  1. Tie it to evidence: Match each AI claim to a raw log, an actual IOC, a verifiable CVE record, or the code itself. Any claim whose source cannot be cited cannot be included in the report. Use AI to attract attention, not as evidence.
  2. Independently check: Also examine areas that the AI ​​calls "clean". A negative AI output is not a guarantee of "no threat"; Never skip your own systematic analysis.
  3. Context filter: Expertly test whether the output fits the organization's architecture, business context, and known normal behavior. "Anomaly" does not always mean "attack".
Caution: Signing an AI-generated incident report without matching every claim with raw evidence carries the same liability as making an accusation without evidence. Smooth output is not accurate output; If a security decision is wrong, the cost is a system crash or a breach missed.

Privacy and ethics: log data is sensitive data

Log records contain usernames, IP addresses, internal server names, file paths and sometimes personal data. They are protected under KVKK (Personal Data Protection Law) in Türkiye and GDPR in Europe; In addition, these are "internal intelligence" that reveal the attack surface of the institution. Pasting an event with the raw log, real IPs, and internal server names into a public AI tool not only exposes personal data but also carries a useful network map to the external server. The rule is simple: anonymize and mask first. Replace real IPs, usernames, internal hostnames with placeholders; If possible, choose corporate tools that have a data processing agreement and do not use your data in model training.

The ethical boundary is at least as important as the technical boundary. The difference between finding a vulnerability and exploiting it without permission is the difference between legal and criminal. In this module, you use AI only in systems for which you are authorized, for defensive purposes and with written permission. Asking AI to do things like "write an attack tool", "how do I infiltrate that site", "produce a working malware" is outside the profession, and modern AI tools reject them anyway.

three mini cases

Case 1 — Safe use. An analyst encounters 1,200 alarms in the SIEM during a night shift. Has the AI ​​summarize raw alerts (anonymized); AI collapses 1,200 alarms into 18 clusters and pulls forward a "340 failed logins from the same internal IP, followed by 1 success" pattern. The analyst verifies this cluster with the raw log, finds a real password brute force attack, and locks the account in 9 minutes. AI accelerated sorting; The analyst made the decision and verification.

Case 2 — Unverified output trap. Another analyst has the AI ​​prioritize a list of vulnerabilities. The AI ​​says "CVE-2024-99999 is critical, patch it now." The analyst plans to patch but never opens the CVE record; whereas there is no such CVE — the model made up the number. The team loses hours chasing a patch that doesn't exist, while the real critical vulnerability is delayed. Verification is omitted, the claim is not linked to the source.

Case 3 — Breach of confidentiality. To speed up an incident investigation, an expert pastes the raw firewall log—with actual internal IPs, usernames, and VPN server names—into a public AI tool. The organization's network topology, naming scheme and user list have gone to an external server. The correct way was to mask the IPs and names and share only the pattern.

Weak prompt / Strong prompt

Weak prompt:

Is there an attack in the following log: 10.2.14.7 user ahmet.yilmaz entered the VPN, then connected to the file server FS-MUHASEBE-01. Also prioritize these vulnerabilities.

This request is flawed in three ways: the real IP, user and server name are shared (privacy violation), the role and boundaries of the AI ​​are not defined, and verifiable evidence is not requested. AI fills in the gaps with guesswork and the risk of fabrication arises.

Powerful prompt:

Your role: DRAFT assistant to the SOC analyst. Decision making; Declare the incident as an "attack", isolate the system, or block the user. Just analyze the anonymous log pattern I gave you. For each claim, indicate which log line you base it on; Mark "[analyst verify]" where you are not sure; spoofing IOC, CVE or IP. Anonymous incident: USER_A accessed VPN via YURTDISI_IP at 03:14; then accessed 4,000 files to the internal file server; The user normally works between 09:00-18:00. Questions: (1) what patterns are suspicious, (2) what additional log evidence should I look for, (3) could there be false positives?

The strong will is anonymous, defines role and boundary, questions attachment to evidence and the possibility of false positives, and prohibits fabrication.

Copiable prompt templates

ROLE AND BOUNDARY DESCRIPTION TEMPLATEYour role: assistant to security analyst preparing DRAFT/ANALYSIS. You are not an analyst; Declaring the incident as an attack, isolating the system, blocking the user or finalizing an official report. The final decision and signature is with the analyst. Show evidence (log line, IOC, CVE, code) for each claim; Mark something that has no evidence as "[must be verified]", do not make it up. Task: [write task].

ANONYMIZATION CONTROL TEMPLATEExtract real IP addresses, user names, internal host/server names, e-mail and domain names, corporate information from the following security data; replace with consistent placeholders (USER_A, IC_IP_1, HOST_1).Keep only the pattern necessary for analysis. Notify me of changes in a list. Data: [paste data]

VALIDATION CHECK TEMPLATEFor each finding you produce, write next to it: (1) what evidence is it based on, (2) what raw record/source should I open to verify, (3) the likelihood of a false positive and why. Use "possible/suspect" when necessary rather than precise language. Non-existent CVE/IOC/IP fabrication.

RISK LEVEL ALLOCATION TEMPLATE Categorize the security assignment I will assign and write justification: (A) low risk - AI outline/summary sufficient, (B) medium risk - analyst must verify, (C) high/very high risk - decision/isolation/notification belongs to the analyst, AI is only helpful. Task: [write task].

Common mistakes

  • Mistaking AI for an analyst. AI scans for patterns but has no responsibility or authority; You decide. The output is a draft, not a verdict.
  • Sharing real IP, user and host name. This is both a KVKK violation and a network map leak that will benefit the attacker; mask first.
  • Relying on negative AI output and relaxing the search. "No threat" doesn't really mean there isn't; Never skip your own systematic analysis.
  • Using made-up CVE/IOC without verification. Can match model number and indicator; Confirm each with official source.
  • Unauthorized/offensive use. Work only defensively, on your own systems, with written permission; Otherwise it is both illegal and unethical.
Tip: Ask yourself one question for each task: “What happens if this output is wrong?” If the answer is “an attack escapes” or “business interruption occurs” — as it often does in security — use AI only for the summary/suggestion/outline and never skip verification.

In summary

Artificial intelligence is a powerful assistant in cybersecurity: it summarizes the log, sorts the alarm, analyzes phishing, scans the code, generates draft reports. But this is a security-critical area; It is up to the qualified expert to declare an incident an attack, isolate a system, block a user, and file an official report. The role of AI in the five stages of the process (detection, analysis, intervention, remediation, reporting) varies depending on the level of risk; As the risk increases, human approval grows. Three disciplines guard each step: evidence, independent check, context filter. And underneath it all, there are two limits: confidentiality (exporting raw data without anonymization) and ethics (authorized, defensive, authorized use only).

Application task

Select three tasks from your own organization (or an example scenario): one low risk (e.g. daily alert summary), one medium risk (e.g. a phishing analysis), one very high risk (e.g. decision to isolate a system). For each, (1) describe the role of the AI ​​in one sentence, (2) write down what verification step you will take, (3) indicate how you will anonymize the data. Then adapt the “Role and Boundaries Definition” template to your medium-risk task, write a prompt, and note how you will verify its output with raw evidence.

checklist

  • [ ] I determined the risk level (low/medium/high/very high) of the task.
  • [ ] I limited the AI's role to "assistant/summary/suggestion/draft"; The decision and signature is with the analyst.
  • [ ] I anonymized the data; real IP, user, host and domain names are masked.
  • [ ] I promised to verify every claim with raw evidence (log, IOC, CVE, code).
  • [ ] Despite the negative AI output, I will conduct my own systematic analysis.
  • [ ] Knowing that it may be fake CVE/IOC/IP, I will confirm it with the official source.
  • [ ] I am limited to authorized, defensive and written authorized use only.