A founder building a clinical decision-support tool gets a question from her lead Series A investor: “How do you handle the AI Safety Institute?” She has heard of the AISI — the Australian Artificial Intelligence Safety Institute that became operational in early 2026 — but she has no idea what the right answer is. Is it a regulator she needs to register with? An evaluator she needs to submit her model to? An optional partnership? None of the above?
The honest answer is “none of the above, but the question still matters.” The AISI is not a regulator. It does not licence models, certify products, or enforce rules. What it does is sit at the centre of a rapidly tightening Australian framework that combines the Voluntary AI Safety Standard, the proposed mandatory guardrails for high-risk settings, sectoral regulators like the OAIC and ASIC, and existing technology-neutral laws — and that framework is what investors and customers are starting to ask startups about.
This article unpacks what the AISI actually is, what “high-risk AI” means in Australia, and what founders building in the AI stack should be doing right now.
What the AISI Is — and Is Not
The AISI was established under the National AI Plan released by the Department of Industry, Science and Resources in December 2025, with initial funding of $29.9 million. It became operational in early 2026 and is positioned as Australia’s contribution to the International Network of AI Safety Institutes, joining counterparts in the United Kingdom, the United States, Japan, Singapore, Canada, the European Union, and others.
Its mandate is narrow but important: pre-deployment safety testing of advanced AI systems, upstream risk assessment, downstream harm analysis of deployed systems, and the provision of independent technical advice to ministers and regulators. Its priority risks are the ones that have driven the international AISI network — chemical, biological, radiological and nuclear (CBRN) uplift, enhanced cyber capabilities, loss-of-control scenarios in frontier models, and information-integrity and influence risks at scale.
What the AISI is not:
- It is not a regulator. It cannot issue fines, suspend products, or compel disclosure outside voluntary engagement.
- It is not a certifier. There is no “AISI tick” that a model can earn and display.
- It is not a one-stop AI authority. Existing regulators retain their jurisdiction — the OAIC for privacy, the ACCC for consumer law and unfair contract terms, ASIC for financial services, the eSafety Commissioner for online safety, the Therapeutic Goods Administration for medical devices, and so on.
For founders, that division of labour matters. The AISI may evaluate a model and publish findings, but the legal consequences of getting AI wrong flow through the existing statutory regulators — and through the courts, where the Australian Consumer Law, the Privacy Act 1988 (Cth), and tort principles continue to bite regardless of whether AISI has ever looked at the product.
What “High-Risk AI” Means in Australia
The phrase “high-risk AI” is used loosely in industry. In Australia, it has a specific legal lineage. The September 2024 Introducing Mandatory Guardrails for AI in High-Risk Settings proposals paper from the Department of Industry, Science and Resources put forward two possible definitions: a principles-based test (focusing on the context of use, capability of the system, and potential for harm to individuals, groups, and society) and a list-based test (enumerating sectors and use cases). Heading into mid-2026, the principles-based approach appears to be winning.
Under the principles approach, indicators of “high-risk” use include:
- Adverse impact on a person’s human rights as recognised in Australia
- Impact on a person’s physical or mental health or safety
- Legal effects, defamation or material disadvantage to a person
- Adverse impact on groups of individuals or broader Australian society
- Adverse impact on the economy, environment, rule of law, or democratic processes
- Models with capabilities that present systemic risk (most relevant to general-purpose models above a compute threshold)
Translated for startup founders, the high-risk lens captures more than the frontier-model use case people associate with the AISI. A startup building credit decisioning, employment screening, biometric matching, healthcare triage, education assessment, insurance pricing, or essential services eligibility tooling is squarely in the high-risk conversation — even if its underlying model is a small fine-tune of an open-source base.
The Ten Guardrails and Where Founders Sit
The Voluntary AI Safety Standard (VAISS), released in September 2024 and now operating alongside the broader Guidance for AI Adoption published with the National AI Plan, sets out ten guardrails that apply across the AI supply chain. The mandatory-guardrails proposal would convert nine of these into legal obligations for high-risk settings (with the tenth replaced by a conformity assessment requirement). The ten guardrails are:
- Establish, implement, and publish an accountability process — including governance, internal capability, and a regulatory compliance strategy
- Establish and implement a risk management process to identify and mitigate risks
- Protect AI systems and implement data governance to manage data quality and provenance
- Test AI models and systems to evaluate performance, and monitor systems after deployment
- Enable human control or intervention to achieve meaningful human oversight
- Inform end-users about AI-enabled decisions, interactions, and AI-generated content
- Establish processes for people impacted by AI systems to challenge use or outcomes
- Be transparent with other organisations across the AI supply chain about data, models, and systems to help them effectively address risks
- Keep and maintain records to allow third parties to assess compliance
- Engage stakeholders and evaluate their needs and circumstances, with a focus on safety, diversity, inclusion, and fairness
For a startup, two things stand out. First, the obligations attach to both developers (those who design, train, or build AI systems and components) and deployers (those who integrate or supply them to end-users). A SaaS startup that wraps a third-party foundation model is a deployer, not a developer — and most of the guardrails still apply. Second, the guardrails are written at the level of process, not outcome. A founder who can show a real risk-management process, real testing, real human-oversight points, and real documentation will satisfy the substantive expectation. A founder relying on “we’ll handle it when it comes up” will not.
Where the Mandatory Layer Actually Sits
A common misconception is that Australia does not yet regulate AI. It does — through technology-neutral laws that already apply with full force. The Privacy Act 1988 (Cth) governs personal information processed by AI systems and is itself being reformed to address automated decision-making and a general right to challenge automated decisions. The Australian Consumer Law in Schedule 2 of the Competition and Consumer Act 2010 (Cth) catches misleading or deceptive AI-driven claims and unfair contract terms in standard-form B2B and consumer contracts. The Online Safety Act 2021 (Cth) reaches AI-generated harmful content. The Anti-Discrimination Act regimes in each state, and the federal Sex Discrimination Act 1984 and Disability Discrimination Act 1992, reach AI hiring and decisioning tools. Sectoral regulators — ASIC for financial services, the TGA for medical devices that incorporate AI as software-as-a-medical-device, APRA for prudentially regulated entities — overlay further requirements.
The AISI sits above this regulatory mosaic as a technical intelligence function. When ASIC or the OAIC needs to understand whether a particular model class poses a particular kind of risk, the AISI is the body designed to provide that analysis. For founders, that means AISI findings can feed directly into enforcement decisions by other regulators — even though the AISI itself does not enforce.
Engagement Pathways That Matter
There are three practical ways a startup intersects with the AISI:
Voluntary safety evaluation. Frontier-model developers and developers of capable general-purpose systems can engage AISI for pre-deployment evaluation. For most Australian startups, this is not relevant — the threshold is high. But for the small group of Australian companies training or substantially modifying base models, engagement is now an expected step before commercial release into sensitive use cases.
Incident reporting and information sharing. AISI is expected to operate channels for reporting AI safety incidents and near-misses, modelled on equivalents in aviation safety. Startups that experience material safety failures in deployed systems should expect (and benefit from) reporting pathways into AISI as well as any sectoral regulator.
Standards and guidance development. AISI’s technical work feeds into national and international standards. Startups that want a seat at the table on how rules are written can engage through the National AI Centre and industry consultations. For deep-tech founders, this is an underrated channel.
What Founders Should Be Doing Now
Map yourself to the high-risk indicators. Apply the proposed principles-based test to your use case. If any of the indicators applies, treat your startup as operating in a high-risk setting for planning purposes — even if mandatory guardrails are not yet in force.
Adopt the ten guardrails as a checklist. Build the accountability process, risk register, testing protocol, human-oversight points, transparency disclosures, and records framework now, at the size where the cost of doing so is small. The marginal cost of bolting these on at Series B is meaningful; the marginal cost of doing so before product-market fit is close to zero.
Get the documentation right. Investors performing AI-specific due diligence in 2026 are starting to ask for model cards, data provenance documentation, evaluation reports, and incident logs. Treat these as first-class artefacts of the company, not afterthoughts.
Watch the mandatory guardrails legislation. The mandatory guardrails proposal remains in policy development and may yet land as standalone legislation, amendments to existing regulators’ statutes, or both. Founders building in high-risk verticals should track the consultation cycle and be prepared to participate.
Don’t conflate AISI with the regulator. A clean conversation with AISI does not insulate a startup from OAIC, ACCC, ASIC, or sector-specific enforcement. The legal risk lives in the existing statutes; AISI is the technical conscience sitting alongside them.
The Bottom Line
The AISI is not an obstacle for most startups. It is a piece of national infrastructure that will, over time, shape how high-risk AI is built, tested, and trusted in Australia. The founders who win in this environment are the ones who treat the guardrails as good engineering practice, document their processes early, and engage with the framework as a feature of doing serious AI work — not as a tax to be minimised.
The Series A investor’s question — “How do you handle the AI Safety Institute?” — is really a proxy for a deeper one: Have you built this company in a way that survives contact with the rules that are coming? The companies that can answer yes will be the ones that get funded, acquired, and trusted with the use cases that matter.
Viridian Lawyers advises Australian startups and technology companies on AI regulation, governance, and commercial deployment. If you are building in a high-risk AI setting and need help mapping your obligations, get in touch.
