Skip to main content

Chapter 3 — Auditing IS Acquisition and Development

The acquisition and development of information systems introduces substantial change and substantial risk. Projects fail; budgets overrun; security vulnerabilities ship; controls are bypassed during implementation; testing is inadequate; documentation is lost. The IS auditor examines project management discipline, feasibility analysis quality, development lifecycle controls, and security testing rigour — assessing whether changes are introduced with appropriate governance, risk consideration, and control. This chapter covers each, with the practitioner perspective on how to plan and execute audits of acquisition and development activities, the specific tests that are conducted, the evidence that is gathered, and the findings that typically emerge.

3.1 Project management audits​

Why project management matters​

IT projects have notable failure rates. Standish Group's CHAOS reports, while methodologically debated, consistently show substantial percentages of projects failing or significantly underperforming. Common causes include unclear requirements, poor estimation, weak governance, scope creep, inadequate testing, and insufficient change management.

Project management discipline — applied with appropriate rigour — substantially reduces failure rates. Audit examines whether discipline is applied.

Project management frameworks​

PMBOK (Project Management Body of Knowledge). PMI's framework; ten knowledge areas, five process groups, forty-nine processes.

PRINCE2. UK origin; widely adopted globally; structured method with defined principles, themes, processes.

Agile frameworks. Scrum, Kanban, SAFe (Scaled Agile Framework), LeSS (Large-Scale Scrum), various others.

Hybrid approaches. Combining structured and agile elements.

The auditor's role isn't to prescribe a framework but to assess whether whatever methodology is used is applied with discipline.

Project audit scope​

A project audit may cover:

Pre-project. Business case, approval, governance setup.

Initiation. Charter, planning, team formation.

Planning. Scope, schedule, budget, risk, communication, procurement plans.

Execution. Performance, quality, risk management.

Monitoring and control. Status reporting, change control, issue management.

Closure. Acceptance, transition, lessons learned, knowledge management.

Timing of audit varies — concurrent with project, after specific phases, after closure.

Pre-project audit​

For business case and approval:

Business case quality. Realistic benefits, costs, risks, alternatives.

Approval authority. Approved at appropriate level.

Strategic alignment. Project supports strategy.

Resource availability. Resources actually available.

Independent review. Where applicable.

Project charter audit​

Charter examined for:

Clear objectives. Specific, measurable.

Scope definition. What is in/out.

Stakeholder identification. Who is involved.

Authority delegation. Project manager authority.

High-level requirements. Sufficient for planning.

Constraints and assumptions. Documented.

Approval. Formal approval recorded.

Project planning audit​

Plans reviewed for:

Comprehensiveness. All necessary plan elements.

Realistic estimates. Schedule and budget reasonable.

Risk identification. Material risks identified.

Risk treatment plans. Specific.

Resource plans. People, equipment, services.

Procurement plans. Where external resources needed.

Communication plans. How stakeholders engaged.

Quality plans. How quality assured.

Change management plans. How changes handled.

Execution and monitoring audit​

During execution:

Progress vs plan. Realistic tracking.

Issue management. Issues identified, tracked, resolved.

Risk management. Risk register maintained; treatments applied.

Change management. Changes formally approved.

Quality assurance. QA activities executed.

Status reporting. Reports to governance forums.

Decision tracking. Decisions documented.

Document management. Project documentation maintained.

Project audit findings​

Common findings:

Scope creep. Scope expanded without formal approval.

Schedule slippage. Behind schedule; recovery unclear.

Budget overrun. Costs exceed plan.

Resource issues. Insufficient or misallocated.

Risk realisation. Risks materialised; treatment failed.

Inadequate testing. Testing insufficient.

Documentation gaps. Documentation incomplete.

Change control weakness. Changes not following process.

Stakeholder disengagement. Key stakeholders not engaged.

Quality issues. Deliverables below expectations.

Project audit reporting​

Reports to:

Project sponsor. Detail.

Steering committee. Summary.

Programme management. Aggregated across projects.

Audit committee. Material findings.

Recommendations specific to project rather than generic.

3.2 Feasibility analysis review​

Feasibility analysis​

Feasibility analysis is the systematic evaluation of whether a proposed initiative is viable across dimensions including technical capability, operational fit, economic justification, schedule reasonableness, legal compliance, and organisational readiness — providing the basis for informed go/no-go decisions before substantial investment is committed.

The auditor examines feasibility analysis quality both as part of project audits and as standalone reviews for major investments.

Feasibility dimensions​

Technical feasibility. Can it be built with available technology and skills?

Operational feasibility. Will it work in the operational environment?

Economic feasibility. Do benefits justify costs?

Schedule feasibility. Can it be delivered in available timeframe?

Legal feasibility. Will it comply with applicable laws and regulations?

Organisational feasibility. Is the organisation ready to absorb and operate it?

Cultural feasibility. Will users adopt it?

Each examined separately; overall conclusion based on all.

Auditor's feasibility review​

For each dimension:

Technical. Is the technology proven? Is the skill base available? Are dependencies achievable?

Operational. Does it fit operating model? Are operational impacts considered? Are operational requirements addressed?

Economic. Are benefits realistic and quantified? Are costs comprehensive? Is timing reasonable? Is sensitivity analysis performed?

Schedule. Is the schedule realistic? Are dependencies considered? Is contingency included?

Legal. Are applicable laws identified? Is compliance addressed? Are licensing issues considered?

Organisational. Is leadership commitment evident? Are resources committed? Is change management considered?

Cultural. Is user impact assessed? Is adoption strategy defined?

Economic feasibility detail​

The economic dimension typically receives most scrutiny:

Cost-benefit analysis. Quantified costs and benefits.

Net Present Value (NPV). Discounted future cash flows.

Internal Rate of Return (IRR). Discount rate making NPV zero.

Payback period. Time to recover investment.

Total Cost of Ownership (TCO). Comprehensive long-term cost.

Return on Investment (ROI). Benefits as percentage of investment.

Sensitivity analysis. How conclusions change with assumption variations.

The auditor verifies analytical rigour and reasonableness of assumptions.

Common feasibility analysis weaknesses​

Patterns the practitioner identifies:

Optimistic estimates. Benefits inflated; costs understated.

Missing categories. Some costs not included (training, integration, ongoing operation).

Weak alternatives analysis. Single option promoted without comparison.

Insufficient sensitivity. Single scenario rather than range.

Bias toward proceeding. Analysis serves justification rather than informing decision.

Inadequate due diligence on vendors. Vendor claims taken at face.

Schedule optimism. Standard project optimism.

Integration complexity underestimated. Connection points underweighted.

Vendor selection audit​

When IT acquisition involves vendor selection:

Requirements completeness. All material requirements specified.

Evaluation methodology. Documented criteria.

Bid evaluation. Per documented criteria.

Reference checks. Performed and documented.

Negotiation transparency. Process documented.

Contract terms. Reviewed for completeness and adequacy.

Due diligence on vendor. Financial, operational, security.

Conflict of interest. Selection panel independence.

For Nepali public sector, Public Procurement Act provisions add specific requirements; PPMO oversight applies.

Make vs buy analysis​

For development vs acquisition decisions:

Skills available. Internal capability for development.

Time to value. Build vs deploy.

Total cost. TCO for each option.

Strategic importance. Whether differentiating capability.

Vendor risk. Dependency considerations.

Maintenance and evolution. Long-term implications.

Audit examines whether analysis was comprehensive.

3.3 SDLC and Agile controls testing​

SDLC overview​

The Software Development Lifecycle is the structured process by which software is conceived, designed, developed, tested, deployed, and maintained — providing the framework within which security controls are integrated throughout development rather than retrofitted afterward, with substantial variation in specific methodologies (waterfall, agile, DevOps, hybrid) but consistent fundamental needs for requirements, design, implementation, testing, deployment, and operation.

The auditor examines controls at each phase regardless of specific methodology.

Waterfall SDLC controls​

Traditional sequential model:

Requirements phase. Requirements gathered, documented, approved.

Design phase. Architecture and detailed design.

Implementation phase. Coding.

Testing phase. System and user acceptance testing.

Deployment phase. Production rollout.

Maintenance phase. Ongoing operation.

Each phase has formal gate; signoff required before proceeding.

Agile SDLC controls​

Iterative incremental approach with multiple variants:

Scrum. Sprints, product backlog, sprint backlog, ceremonies (planning, daily standup, review, retrospective), roles (product owner, scrum master, development team).

Kanban. Continuous flow; WIP limits; visualisation.

SAFe. Scaled framework for multiple teams.

XP (Extreme Programming). Specific practices (pair programming, TDD, continuous integration).

Lean. Waste elimination focus.

Controls in Agile differ from waterfall but exist:

Definition of done. Quality criteria for stories.

Sprint review and retrospective. Quality and process review.

Continuous integration. Automated build and test.

Code review. Peer review of changes.

Automated testing. Reduces defect escape.

Pipeline gates. Quality gates in CI/CD.

Stakeholder engagement. Product owner involvement.

DevOps and DevSecOps​

DevOps integrates development and operations; DevSecOps adds security:

Continuous integration. Code merged and built frequently.

Continuous delivery. Code deployable continuously.

Continuous deployment. Code deployed automatically.

Infrastructure as code. Infrastructure managed through code.

Security as code. Security controls codified.

Automated testing. Throughout pipeline.

Monitoring and feedback. Production telemetry feeding development.

Control testing in SDLC​

For each phase, specific tests:

Requirements.

  • Are functional requirements documented?
  • Are non-functional requirements (security, performance, availability) included?
  • Are requirements approved by appropriate stakeholders?
  • Are security requirements specifically identified?
  • Are compliance requirements identified?

Design.

  • Is architecture documented?
  • Has security architecture review occurred?
  • Are threat models created for sensitive systems?
  • Is design review evidence available?

Implementation.

  • Is code stored in version control?
  • Is code review performed and documented?
  • Are coding standards defined and followed?
  • Are secure coding practices applied?
  • Are secrets management practices followed?

Testing.

  • Are test plans documented?
  • Are tests executed per plan?
  • Are test results documented?
  • Are security tests performed (SAST, DAST, dependency scanning)?
  • Are penetration tests conducted for sensitive systems?
  • Are UAT signoffs documented?

Deployment.

  • Are deployment procedures documented?
  • Are deployment approvals obtained?
  • Are rollback procedures defined?
  • Are deployment activities logged?
  • Are post-deployment verifications performed?

Maintenance.

  • Is change management process applied?
  • Are patches applied per schedule?
  • Are monitoring and alerting configured?
  • Is documentation maintained?

SDLC audit findings​

Common findings:

Requirements gaps. Non-functional requirements missing or weak.

Security requirements absent. Functional requirements without security counterparts.

Inadequate design review. Review evidence missing.

Code review gaps. Reviews documented but cursory; or skipped under pressure.

Test gaps. Testing incomplete; security testing absent; UAT cursory.

Documentation gaps. Documentation missing or out of date.

Change control bypasses. Changes deployed without approval.

Inadequate segregation of duties. Same person developing and deploying.

Missing approval evidence. Verbal approvals; missing signoffs.

Agile audit specifics​

For Agile environments:

Definition of done quality. Includes security and quality criteria.

Retrospectives substantive. Real learning vs ritualistic.

Pipeline maturity. CI/CD with appropriate gates.

Test automation. Adequate coverage.

Stakeholder engagement. Product owner participation real.

Backlog management. Quality of backlog.

Sprint discipline. Sprint commitments and outcomes.

Technical debt. Tracked and addressed.

The auditor adapts approach to Agile reality — fewer document-heavy artefacts; more operational evidence.

Segregation of duties​

Key control across SDLC:

Development and production. Developers don't have production access.

Code and approval. Different people commit and approve.

Deployment and audit. Different people deploy and verify.

Privileged access management. Where SoD impossible, compensating controls (logging, review).

For smaller Nepali organisations, complete SoD often impractical; compensating controls (close supervision, logging, periodic review) essential.

3.4 Security testing methods​

Security testing in audit​

The auditor examines whether security testing is performed and effective. Common testing methods:

SAST — Static Application Security Testing​

Static Application Security Testing analyses source code, bytecode, or binary without executing the application, identifying security vulnerabilities through pattern matching, data flow analysis, and other techniques, providing early detection during development before code reaches runtime.

Tools:

  • Commercial. Veracode, Checkmarx, Fortify (Micro Focus/OpenText), SonarQube (paid editions).
  • Open-source/freemium. Semgrep, SonarQube Community, Bandit (Python), Brakeman (Ruby).
  • GitHub native. GitHub Advanced Security CodeQL.
  • GitLab native. GitLab SAST.

Strengths:

  • Early detection.
  • Full code coverage possible.
  • No runtime environment needed.

Weaknesses:

  • False positives common.
  • Limited business logic understanding.
  • Configuration issues missed.

DAST — Dynamic Application Security Testing​

Dynamic Application Security Testing exercises a running application from the outside — sending requests and analysing responses — to identify vulnerabilities exploitable at runtime, simulating attacker techniques against a deployed application.

Tools:

  • Commercial. Burp Suite Pro, Acunetix, Netsparker (Invicti), AppScan.
  • Open-source. OWASP ZAP, Nuclei.

Strengths:

  • Tests deployed application.
  • Identifies runtime issues.
  • Configuration issues caught.

Weaknesses:

  • Coverage depends on traversal.
  • Can disrupt application under test.
  • Slower than SAST.

IAST — Interactive Application Security Testing​

Combines SAST and DAST approaches; instruments application code and observes during testing.

Tools: Contrast Security, Checkmarx CxIAST, others.

SCA — Software Composition Analysis​

Software Composition Analysis identifies and tracks third-party components in an application — open-source libraries, frameworks, packages — and matches them against vulnerability databases to identify known vulnerabilities, increasingly important as modern applications heavily depend on third-party components.

Tools:

  • Commercial. Snyk, Black Duck, Mend (WhiteSource), JFrog Xray.
  • Open-source/freemium. Dependabot (GitHub), npm audit, pip-audit, OWASP Dependency-Check.

Modern applications use many dependencies (a typical Node.js application may have hundreds in dependency tree). SCA visibility into vulnerability exposure is essential.

Penetration testing​

Penetration testing is the authorised simulated attack against a system, network, or application — conducted by skilled testers following established methodologies — to identify and demonstrate exploitable vulnerabilities, with results provided to the system owner for remediation, distinct from vulnerability scanning in that it includes exploitation rather than mere identification.

Covered extensively in ENCTNS615 Chapter 3. For audit context, penetration test reports are evidence of security testing.

The auditor examines:

Test scope. Appropriate for system criticality.

Test methodology. Documented and rigorous.

Tester qualifications. Appropriate certifications and experience.

Findings comprehensiveness. Specific, with evidence.

Severity ratings. Reasonable.

Remediation tracking. Findings addressed.

Retest. Verification of remediation.

Cycle. Appropriate frequency.

Other testing methods​

Fuzzing. Random/malformed input to identify crashes and vulnerabilities.

Threat modelling. Systematic identification of threats during design.

Configuration assessment. Specific configuration testing.

Compliance scanning. Testing against specific frameworks.

Red team exercises. Adversarial simulation more extensive than pentests.

Bug bounty programmes. External researcher engagement.

Security testing programme audit​

Beyond individual tests, the auditor assesses the programme:

Programme governance. Who decides what to test when.

Coverage adequacy. Material systems covered.

Frequency appropriateness. More frequent for higher risk.

Tool deployment. Appropriate tools in pipelines.

Findings management. Identified issues tracked.

Remediation discipline. Issues addressed in reasonable time.

Effectiveness measurement. Programme outcomes assessed.

Continuous improvement. Lessons applied.

Common security testing findings​

Insufficient coverage. Some systems not tested.

Test gaps. Methods not used (e.g., no SAST in pipeline).

False sense of security. Tests performed but findings not addressed.

Tools deployed but not tuned. High false-positive noise; real findings lost.

Tests not integrated. Security testing separate from development.

Findings not tracked. Issues identified but not remediated.

Remediation theatre. Issues "addressed" without actual fix.

No retest. Remediation not verified.

Nepal context for security testing​

For Nepali banks and major enterprises:

Maturity varies.

  • Top-tier banks have integrated security testing programmes.
  • Mid-tier institutions implementing.
  • Smaller organisations often ad-hoc.

Tool adoption.

  • SAST/DAST tools deployed at major institutions.
  • SCA increasingly common.
  • Penetration testing standard for material systems.

Local capability.

  • Several Nepali pentest firms.
  • International firms for higher-tier engagements.
  • Internal teams at largest institutions.

Findings reality.

  • Findings volume substantial.
  • Remediation challenges common.
  • Pressure to focus on highest-risk.

For MSc graduates, security testing capability is valuable; combining SAST/DAST tool expertise with traditional pentest skills creates strong professional profile.

From acquisition and development to operations​

Once systems are built and deployed, they enter operations. The next chapter takes up IS operations and business continuity — the steady-state operational disciplines and the resilience disciplines that enable recovery from disruption. The 12-mark weight reflects the breadth and depth of this area, central to most IS audit engagements.

· min read