Chapter 5 — Social Engineering and Phishing Attacks

Social engineering — manipulating people to divulge information or take actions that compromise security — has always been among the most effective attack vectors. Phishing, the email-based variant, is the most common entry point for cyberattacks affecting organisations of every size. Generative AI has transformed this landscape: what previously required either generic templates or skilled human writers can now be produced at scale, in any language, customised for any target, in seconds. The defensive response combines technical controls, organisational practices, and user education adapted to the new threat reality. This chapter examines AI-generated phishing and spear-phishing, AI-crafted social engineering more broadly, and the defensive measures that mitigate these threats.

5.1 Generative AI for phishing content generation and spear-phishing attacks

Phishing overview

Phishing is the fraudulent practice of sending communications — typically email, but also SMS (smishing), voice (vishing), social media, and other channels — that appear to come from legitimate sources to induce recipients to reveal credentials, financial information, or other sensitive data, or to take actions (clicking malicious links, opening malicious attachments, transferring funds) that compromise security.

Phishing has been the dominant attack vector for over two decades. The 2024 Verizon DBIR and similar industry reports consistently identify phishing and pretexting as primary attack vectors for breaches.

Phishing taxonomy

Bulk phishing. Generic messages sent to many recipients. Low effort; low success rate per recipient; high volume.

Spear phishing. Targeted messages crafted for specific recipients. Higher effort; higher success rate.

Whaling. Spear phishing targeting executives. Highest effort and value.

Business Email Compromise (BEC). Sophisticated impersonation typically of executives to drive financial transfers.

Vendor email compromise. Compromised legitimate vendor accounts used to defraud customers.

Clone phishing. Legitimate previous emails duplicated with malicious links/attachments substituted.

Reply chain hijacking. Inserting malicious content into existing email threads.

Pre-AI phishing constraints

Traditional phishing faced trade-offs:

Quality vs scale. Hand-crafted spear phishing was effective but limited in volume. Bulk phishing was high-volume but low-quality (obvious to attentive recipients).

Language coverage. Quality content in many languages required many writers.

Personalisation cost. Each personalised message required time.

Detection patterns. Repetitive content generated by templates was detectable.

AI changes the equation

Generative AI removes these constraints:

Scale + quality. AI produces high-quality content at scale.

Multilingual. Same prompt produces content in any language.

Personalisation automation. Personal details from reconnaissance integrated automatically.

Pattern variability. Each message can be unique.

Voice and tone matching. AI matches communication style of impersonated individuals.

Specific AI-phishing techniques

Persona crafting. AI generates plausible sender personas with backstory, communication style, social media presence.

Content generation. Email body, subject line, signature crafted for context.

Pretext development. Believable reason for the communication generated.

Localisation. Cultural references, holidays, current events appropriate to target's context.

Language matching. Quality of writing matched to expected sender (executive vs employee vs vendor).

Domain spoofing assistance. AI suggests look-alike domains, typosquatting variations.

Landing page generation. AI generates phishing landing pages convincingly mimicking legitimate sites.

Conversational follow-up. AI handles email replies maintaining the deception.

Spear-phishing example (defensive analysis)

A simulated spear-phishing attack against a Nepali bank executive:

Reconnaissance phase (Chapter 2 techniques):

• LinkedIn profile of CFO: education, career path, current responsibilities.
• Social media: family details, hobbies, recent travel.
• News articles: speaking engagements, business activities.
• Public records: company filings, corporate roles.

Pretext development:

• AI generates a pretext: a counterparty CFO of an Indian bank requesting a meeting about cross-border payment cooperation, referencing a public conference both attended.

Content generation:

• Email body in formal English appropriate for executive-to-executive.
• Subject line creating relevance.
• References to mutual acquaintances (identified through public LinkedIn connections).
• Professional signature matching typical Indian bank format.
• Calendar invite with malicious link.

Sending infrastructure:

• Domain similar to legitimate Indian bank.
• Email infrastructure with appropriate SPF/DKIM/DMARC to pass basic filtering.
• Sender persona LinkedIn profile created.

The attack quality is substantially higher than what was previously feasible without skilled human effort. Defensive measures discussed in Section 5.3 must address this elevated threat.

Voice phishing (vishing)

AI voice cloning enables vishing attacks at new levels:

Capability. Voice cloning from short audio samples (15-30 seconds) produces convincing reproductions. Tools: ElevenLabs, Resemble AI, various open-source.

Sources of voice samples. Earnings calls, conference presentations, podcasts, social media videos, phone greetings.

Attack scenarios:

• Executive impersonation requesting urgent transfers.
• IT helpdesk impersonation requesting credentials.
• Family member impersonation in extortion scenarios.
• Customer impersonation for account takeover.

Real-world incidents

**$25M Arup deepfake fraud (2024).** Hong Kong arm of engineering firm Arup defrauded of$25M through deepfake video call where attackers used AI to impersonate executives.

Various smaller incidents. Multiple reported cases of executive voice impersonation for wire fraud.

Family member vishing. Reported cases of attackers calling family members with cloned voice claiming emergency requiring immediate financial help.

For Nepali context:

• Reports of vishing attacks increasingly common in banking sector.
• Nepali language voice cloning emerging.
• Limited public reporting of AI-specific incidents but underlying patterns evident.

SMS phishing (smishing)

Less affected by AI generation directly (SMS being short) but:

• AI generates SMS content in any language.
• AI-suggested URL shorteners and obfuscation.
• AI personalisation based on telecom and other data.

For Nepali context, smishing is the dominant phishing vector for many consumers — banks, telecoms, government services all targeted.

Multi-channel attacks

Modern sophisticated attacks span channels:

1. Initial reconnaissance via OSINT.
2. SMS to induce engagement.
3. Email follow-up with detail.
4. Phone call to close the deception.
5. Final exploitation.

AI assists each channel with appropriate content.

5.2 Crafting social engineering strategies with AI

Social engineering

Social engineering is the manipulation of individuals to take actions or divulge information that compromises security, exploiting human psychology rather than technical vulnerabilities, ranging from simple pretexting to elaborate multi-stage operations.

Phishing is one specific form. Broader social engineering includes pretexting, baiting, tailgating, quid pro quo, and others.

Psychological principles exploited

Social engineering exploits:

Authority. People defer to perceived authority.

Urgency. Time pressure reduces careful evaluation.

Reciprocity. People feel obligated to return favours.

Scarcity. Limited availability creates pressure.

Liking. People comply more with those they like.

Social proof. People follow what others do.

Consistency. People honour prior commitments.

These principles (Cialdini's classics, with subsequent additions) underpin most social engineering. AI exploits them at scale.

AI in social engineering strategy

Target profiling. AI builds psychological profile from public information.

Channel selection. AI suggests most likely effective channel for specific target.

Timing. AI identifies likely timing for maximum effect.

Pretext selection. AI generates pretexts likely to work with specific target.

Content tailoring. AI customises content for target's psychological profile.

Conversation management. AI handles ongoing interaction, adapting to responses.

Multi-stage planning. AI plans extended operations with multiple touchpoints.

Specific attack scenarios

Executive impersonation.

• Target: subordinate of executive.
• Pretext: urgent request from executive.
• Channel: email or text purportedly from executive.
• Goal: wire transfer, credential disclosure, sensitive data.
• AI assistance: voice/style matching, plausible urgency, behavioural prediction.

Helpdesk impersonation.

• Target: end user.
• Pretext: IT support resolving issue.
• Channel: phone, email, or chat.
• Goal: credentials, malware installation, remote access.
• AI assistance: technical-sounding pretext, knowledge of organisation's actual systems.

Vendor impersonation.

• Target: accounts payable or procurement.
• Pretext: legitimate vendor with banking change.
• Channel: email with seemingly legitimate documentation.
• Goal: payment redirection.
• AI assistance: convincing documentation, vendor communication style matching.

Customer impersonation.

• Target: customer service representative.
• Pretext: customer with account issue.
• Channel: phone, email, chat.
• Goal: account takeover, information disclosure.
• AI assistance: realistic customer story, social engineering of verification.

Recruitment scams.

• Target: job seekers.
• Pretext: legitimate-seeming job opportunity.
• Channel: email, LinkedIn, job boards.
• Goal: fee fraud, identity theft, malware delivery.
• AI assistance: convincing company personas, realistic job descriptions, professional communication.

Long-term social engineering

Beyond single-interaction attacks, AI enables sustained operations:

Persona maintenance. AI maintains long-term fake personas with consistent backstory.

Relationship building. Slow cultivation of relationships before exploitation.

Insider recruitment. Convincing insiders to provide assistance (intentional insider threat).

Reconnaissance through engagement. Gathering information through extended interaction.

These attacks are resource-intensive but high-value when successful — typically reserved for high-value targets but increasingly possible at lower target tiers due to AI automation.

Sophisticated multi-stage attacks

A sophisticated attack might unfold:

Week 1. AI creates fake LinkedIn profile of journalist; connects with target executive.

Weeks 2-4. Periodic engagement through likes, comments, brief messages; builds familiarity.

Week 5. AI-generated direct message proposing interview.

Week 6. AI-conducted interview via video call (deepfake of journalist).

Week 7. Follow-up email requesting documents for "fact-checking" — actually for reconnaissance/exploitation.

Week 8+. Information used for targeted attack on target's organisation.

This timeline previously required substantial human time; AI automation enables many such operations in parallel.

Detection challenges

AI-generated social engineering creates detection challenges:

Content quality. Traditional indicators (grammar errors, unusual phrasing) no longer reliable.

Personalisation. Generic-looking phishing easier to spot than personalised.

Volume. AI enables more attacks, more variations.

Adaptation. AI adapts to defensive responses.

Multi-channel. Single-channel detection misses attacks spanning channels.

Awareness challenges

Traditional security awareness training faces challenges:

• Users trained to spot generic phishing don't recognise sophisticated AI-generated.
• Quality of AI content exceeds user discrimination ability.
• Personalised attacks defeat training based on generic examples.

New training approaches needed (Section 5.3).

5.3 Defensive measures — awareness training, email filtering, sandboxing

Defensive layers

The defence is layered:

Email gateway (technical filter)
        ↓
Network controls (URL filtering)
        ↓
Endpoint protection (sandboxing, EDR)
        ↓
User awareness (the human layer)
        ↓
Response procedures (when something gets through)

Each layer catches some attacks; combined they catch most.

Email security gateways

Established vendors:

• Microsoft Defender for Office 365 / Exchange Online Protection.
• Google Workspace security.
• Proofpoint.
• Mimecast.
• Cisco Secure Email.
• Barracuda.
• Trend Micro.
• Vade.

Capabilities:

• Spam filtering.
• Malware scanning.
• URL analysis and rewriting.
• Attachment sandboxing.
• Reputation analysis.
• DMARC/DKIM/SPF verification.
• Anomaly detection.
• AI-based content analysis.

AI in email security

Modern email security gateways use ML extensively:

Sender reputation. ML models score sender reputation.

Content analysis. Models trained on phishing examples classify new emails.

Anomaly detection. Unusual communication patterns flagged.

URL analysis. ML models analyse URLs and landing pages.

Image analysis. Computer vision for image-based threats.

Conversation context. Analysing email threads for impersonation.

Adversarial robustness. Models hardened against AI-generated content.

DMARC, SPF, DKIM

Email authentication standards:

SPF (Sender Policy Framework). Lists authorised sending IPs for domain.

DKIM (DomainKeys Identified Mail). Cryptographic signature on outgoing emails.

DMARC (Domain-based Message Authentication, Reporting, and Conformance). Policy combining SPF and DKIM with reporting.

Properly configured DMARC with reject policy substantially reduces spoofing of the domain.

For Nepali organisations:

• DMARC adoption increasing but incomplete.
• Many .np domains have no DMARC policy.
• Banks and major enterprises mostly compliant.
• Smaller organisations frequently not.

The dmarc.org test tools and various commercial services help organisations evaluate and improve their email authentication posture.

URL analysis and rewriting

Email gateways analyse URLs:

• Reputation checks against known-bad URLs.
• Real-time scanning when clicked.
• Sandboxing of destination.
• URL rewriting routing through inspection proxy.

Tools: Microsoft Safe Links, Proofpoint TAP, Mimecast URL Protect, others.

Attachment sandboxing

Suspicious attachments executed in isolated environments:

• Observe behaviour for malicious activity.
• Allow or block based on behaviour.

Capable sandboxes detect even unknown malware through behavioural analysis. Sophisticated malware may detect sandbox environment and behave benignly.

Behavioural analysis

Beyond content, behaviour analysed:

• Unusual sending patterns.
• New sender to organisation.
• Communication time anomalies.
• Sender impersonation attempts.
• Reply-to mismatches.

User awareness training

Despite technical controls, some phishing reaches users. User education matters.

Training elements:

• Recognising phishing (with current examples).
• Reporting suspicious messages.
• Understanding why care matters.
• Specific organisational procedures.
• Verification procedures (out-of-band confirmation).

Modern approach:

• Continuous rather than annual.
• Phishing simulations to provide practice.
• Personalised based on individual risk.
• Updated regularly with current threat patterns.

Tools:

• KnowBe4 (largest training vendor).
• Proofpoint Security Awareness.
• Cofense (PhishMe).
• Hoxhunt.
• Mimecast Awareness Training.

For Nepali enterprises:

• Major banks have established awareness programmes.
• Phishing simulations increasingly common.
• Local-language content important; Nepali phishing examples more relevant than US ones.
• Mid-size enterprises catching up.

AI-aware awareness training

Training must address AI-era phishing:

• Awareness that quality of phishing has improved.
• Verification procedures regardless of content quality.
• Out-of-band confirmation for sensitive requests.
• Recognising AI-generated voice in calls.
• Understanding multi-channel attacks.
• Reporting culture (no blame for falling for sophisticated attacks).

Defensive automation

Beyond user reporting, automation:

Automated takedown. Of phishing domains and infrastructure.

Automated user notification. When suspicious emails received.

Automated response. When users click suspicious links — credential reset, session termination.

Automated investigation. Of reported phishing.

Specific Nepali bank example

A mature Nepali bank's anti-phishing programme might include:

Technical layer:

• Microsoft Defender for Office 365 with all features enabled.
• DMARC reject policy.
• URL rewriting and sandboxing.
• Attachment sandboxing.
• ML-based anomaly detection.

Awareness layer:

• Quarterly phishing simulations.
• Monthly micro-learning.
• Annual deep training.
• New-employee specific training.
• Executive-specific training for whaling.

Response layer:

• Easy reporting (PhishAlert button).
• 24/7 phishing investigation.
• Automated takedown of identified infrastructure.
• Customer communication for active threats.
• Lessons learned process.

Threat intelligence:

• Continuous monitoring of phishing infrastructure targeting the bank's brand.
• Coordination with NPCERT and peer institutions.
• Customer education through public channels.

The combination produces meaningful reduction in phishing success rate.

Phishing-resistant authentication

The strongest defensive measure: authentication that doesn't rely on user judgement.

FIDO2 / WebAuthn. Hardware-backed authentication that doesn't work with phishing sites (the cryptographic challenge is tied to legitimate domain).

Certificate-based authentication. Cryptographic certificates rather than passwords.

Smart cards. Physical authentication tokens.

Push notifications with number matching. Authentication app shows number; user enters number; prevents simple push-bombing attacks.

These mechanisms eliminate or substantially reduce phishing vulnerability for authentication. Many Nepali banks are moving toward FIDO2 for customer and employee authentication; gradual rollout.

Synthesising the defence

Effective anti-phishing combines:

• Technical controls that block most automated phishing.
• Awareness that helps with sophisticated attacks.
• Resilient authentication that survives phishing of credentials.
• Detection of successful phishing post-event.
• Response that limits damage.

No single layer suffices. Combined layers achieve meaningful protection — though never perfect protection, given the asymmetric advantage attackers retain.

The arms race continues

The phishing arms race will continue. AI advances will produce more capable attacks; AI advances will produce better defences. The overall posture of organisations will depend on:

• Investment in defensive capabilities.
• Adoption of phishing-resistant authentication.
• Quality of user awareness.
• Speed of detection and response.
• Cultural support for security practices.

For Nepali enterprises and the MSc graduate building careers in security, the discipline is permanent — there will not be a "solved phishing" condition in the foreseeable future. Continuous improvement in the face of evolving threats is the operational reality.

The next chapter takes up another offensive application of generative AI — malware development and wireless attacks — examining what AI enables for attackers and what defenders must do in response.