AI Governance + Oversight

Expert Insights from Davis Wright Tremaine

Artificial intelligence is no longer a niche technology. If it hasn’t already, it will soon permeate nearly everything that our organizations, and we as Ethics & Compliance professionals, do. It impacts hiring platforms, enables personalized customer experiences, underpins our supply chains, and drives efficiency and innovation across organizations. But alongside this transformative potential comes substantial (and often novel) legal, ethical, and reputational risks. The challenges of managing these risks are increased by a patchwork regulatory landscape, rapidly changing technology, and the potential for significant harm if AI systems are misused.

The potential consequences of failing to properly govern AI and its use span far beyond regulatory risk. Imagine that your company loses control of the data that it planned to monetize as a central aspect of its business strategy. Imagine that your business spends extensive time and resources to develop an AI system that it then cannot commercialize or sell to key target clients. Imagine that your company is hit with a massive copyright lawsuit that brings not only monetary damages, but a threat to the viability of your AI system. These are not theoretical scenarios, and many ethics and compliance teams are, at best, playing catch up.

Whether your organization is developing or deploying AI systems, the mandate for ethics and compliance professionals is clear. We must apply our skills, cross-functional perspectives, and cross-organizational visibility to tackle the challenges that AI presents. Our ability to identify not only existing legal risks but also those on the horizon; to translate those risks into concrete, practical controls; and to design tailored training and sophisticated governance and monitoring frameworks are crucial to the development of effective AI governance and compliance.

TIP: Listen along to Bill Coffin as he narrates this section. Look for the audio cue in the upper right.

High-Level Overview of the

Regulatory Landscape

The AI regulatory landscape is rapidly developing in numerous jurisdictions in the U.S. and across the globe. As discussed in more detail below, this landscape is a patchwork of state laws regulating various aspects of AI, a handful of U.S. federal rules and enforcement actions, and international legal regimes directed at AI. There also are numerous other broadly applicable legal regimes that apply to AI’s many uses. Keep in mind that if it is not legal for you to do something, it is also not legal for AI to do that same thing on your behalf. Below, we review some of the key relevant compliance considerations, including privacy, data security, antitrust, copyright, and confidentiality, as well as provide additional considerations relevant to government contractors.

AI Governance & Responsible AI

THE BIG PICTURE

Although the U.S. Congress has not enacted a federal law to regulate the development or deployment of AI, the Trump administration has adopted an AI Action Plan designed to maintain U.S. leadership in AI by:

Promoting AI exports
Streamlining the development of infrastructure that supports AI
Mandating “ideological neutrality” in AI models

In addition, federal agencies like the National Institute of Standards and Technology (NIST) have developed AI governance frameworks for companies to use to mitigate and manage AI risks.

ACROSS THE UNITED STATES

California laws require certain developers of generative AI systems to disclose what data was used to train such systems, to create AI detection tools that allow users to determine whether content is AI-generated, and to include “latent” disclosures in such content. California and other states have also enacted laws governing the performance of personal or professional services by AI-generated “digital replicas” of individuals. Utah has enacted laws governing AI chatbots that interact with consumers, imposing rigorous requirements on tools that offer mental health services and prohibiting the unauthorized commercial use of simulated or artificially recreated personal identities, including simulations created using generative AI. At least 38 states have enacted laws governing some aspect(s) of AI. Colorado is the first state thus far to enact a comprehensive law governing AI systems, and others are likely to follow. Among other things, the Colorado AI Act imposes substantial restrictions and compliance obligations on developers and deployers of high-risk AI systems that are intended to interact with consumers and make or be a substantial factor in making “consequential decisions” in such areas as employment, insurance, housing, credit, education, and healthcare. Texas initially introduced legislation modeled on the Colorado AI Act but ultimately enacted a law that restricts the development and deployment of AI for a much narrower set of purposes, including the creation of child pornography and behavioral manipulation.

AI Governance & Responsible AI

The European Union has enacted comprehensive legislation (the EU AI Act) that classifies AI systems according to the risks they pose, prohibiting those deemed unacceptable (e.g., social scoring) and regulating others that are high-risk. Developers of AI systems (called “providers”) have the bulk of the responsibilities, but deployers (called “users”) of such systems have some obligations as well. The Act also regulates developers of large AI models, requiring such entities to comply with European copyright directives and conduct certain evaluations, testing, and incident reporting.

The EU AI Act has extraterritorial reach, capturing AI systems whose outputs are used in the EU. The Act operates in tandem with the EU General Data Protection Regulation, with the latter controlling when there is a conflict between the two. In addition to the EU, other jurisdictions are developing regulatory frameworks, including Singapore, South Korea, China, and Brazil.

Finally, multiple organizations—including non-governmental organizations, government agencies, and companies—have developed principles for “responsible AI.” While these principles may differ in their specifics depending on the particular framework, they are generally designed to promote the human-centric design of AI systems; fairness and elimination of algorithmic bias; accountability and oversight; transparency; privacy and security; and safety and reliability.

Data Privacy

Processing personal data to train, fine-tune, or use AI systems can trigger state, federal, and international data privacy laws.

Personal data is broadly defined as information that is linked or linkable, directly or indirectly, to an identified or identifiable natural person, and it can include everything from images and audio recordings to IP addresses and device identifiers. Developers of AI systems use massive amounts of data to train and fine-tune their systems, and deployers use data to fine-tune models to suit their particular needs. To determine their obligations under privacy laws, both must know whether data they use includes personal data, the source of such data, the legal authority to collect and use the data, and what individuals were told regarding how their data would be used.

Developers and deployers of AI systems may collect data directly from consumers, license data from third parties, or scrape data from websites and other online platforms. Each of these sources present different legal implications under privacy laws and, making things more complicated, privacy

laws differ regarding what data they allow entities to collect from consumers, how they may collect such data, for what purposes they may use the data, and under what circumstances they may disclose the data.

And while state privacy laws exempt “publicly available information” from the definition of “personal data,” they define “publicly available information” differently, making it difficult to know what obligations attach to data that was scraped from the internet for AI model training.

In addition, privacy laws restrict the processing of personal data for “profiling,” which state privacy laws generally define as any kind of automated processing of personal data—such as by predictive or generative AI systems—to evaluate, analyze, or predict personal characteristics, such as health, preferences, interests, or behavior.

Data Security

The proliferation of AI systems has exacerbated existing cybersecurity threats and introduced new ones. Hackers can use AI systems to launch sophisticated cyberattacks (such as deepfake-based social engineering) at a greater scale. AI systems also have become the targets of both familiar and novel cyber threats. Attackers looking to steal troves of sensitive data may target AI systems and their foundation models and may launch AI-specific attacks, such as by injecting manipulated or malicious data into AI models to produce incorrect or harmful outputs.

There are no cybersecurity laws specific to AI systems in the United States. Companies must comply with broadly applicable cybersecurity requirements at both the state and federal levels in securing their AI systems—and in defending against cybersecurity risks posed by AI. At a high level, companies must implement and maintain “reasonable” and “appropriate” cybersecurity controls for AI and other systems. Some state laws have more detailed requirements, such as implementing access controls, system monitoring, and encryption. If an organization's AI system is affected by a breach of personal data or certain significant cybersecurity incidents, they may be required to disclose that incident under various state and federal laws.

In the EU, broadly applicable and comprehensive cybersecurity laws, including the Network and Information Security Directive (NIS2), govern the security of organizations’ networks and information systems—including AI tools. The EU AI Act has requirements related to cybersecurity of “high-risk” AI systems, but they are general. For example, those systems must prevent and respond to malicious attempts to manipulate training data or AI models. In both the U.S. and EU, highly regulated companies in industries like financial services and healthcare face additional, prescriptive requirements.

Numerous authorities have published guidance and risk management frameworks to help companies manage cybersecurity risks associated with the development, deployment, and use of AI systems, including the Federal Bureau of Investigation (FBI), the NIST, the National Security Agency (NSA), the Cybersecurity & Infrastructure Security Agency (CISA), and authorities from Canada, the U.K., Australia, and New Zealand.

Copyright

AI creates significant copyright compliance challenges, particularly related to how models are trained and outputs are used. Training large models often involves ingesting vast datasets that may include copyrighted text, music, images, or code. Without appropriate licenses or reliance on recognized exceptions, companies can face infringement claims. Courts and regulators are still grappling with how to apply fair use and other legal doctrines in the context of AI; initial guidance and decisions have already produced mixed results, and over 40 cases are pending in U.S. courts. All of this has created uncertainty regarding the use of copyrighted works for model training. And the high number of works necessary to train foundation models—as well as the statutory damages scheme—and potential for attorneys’ fee awards under copyright law may make any finding of liability tremendously costly.

AI-generated outputs raise risks.

Outputs that either reproduce or are substantially similar to copyrighted works used in training could also lead to liability for the model deployer. And even where the output is novel, it is not clear whether the output is protected under the various intellectual property law frameworks (for example, the U.S. Copyright Office’s position is that works created by AI systems are generally not eligible for copyright protection, but this issue is less settled in the U.K.).

Antitrust

The rapid adoption of AI raises new antitrust compliance risks. One such concern relates to data access and exclusivity: firms with privileged datasets may foreclose competitors, while data-sharing arrangements risk being construed as collusion.

Market power issues can arise when dominant players leverage control over infrastructure, models, or platforms to either tie services or to block rivals. AI may also heighten collusion risks.

For example, agreements among competitors to contribute competitively sensitive data to algorithmic pricing tools may have the unintended consequence of aligning prices across competitors, and training on sensitive data could facilitate unlawful information exchanges.

Other potential risks include standard-setting collaborations, which should be designed to foster interoperability without excluding rivals, and open-source initiatives should be structured to avoid anti-competitive licensing restrictions. There also is a risk of self-preferencing by AI-enabled platforms, such as prioritizing affiliated services in recommendations or rankings.

Confidentiality

Ensuring the confidentiality of an organization’s sensitive and proprietary information should be a central pillar of any AI governance program. The consequences of a misstep can be severe and can include regulatory penalties and contractual breaches, as well as reputational harm and loss of competitive advantage.

Failure to properly implement confidentiality processes can lead to unauthorized disclosure of sensitive information in unexpected ways.

For example, if trade secrets are used as inputs for an AI system, and those inputs are incorporated into the AI system’s training data without appropriate safeguards, the system could later generate outputs to third parties that inadvertently reveal or replicate the trade secret material. In this scenario, the organization's ability to protect its key intellectual property could be jeopardized.

Government Contractor-Specific Considerations

Organizations that sell, or hope to sell, AI solutions to the federal government, (and those that are subcontractors on federal government contracts) have additional compliance considerations. These include obligations in the Federal Acquisition Regulations and Defense FAR Supplement (which impose strict obligations related to conflicts of interest, procurement integrity, and protection of controlled unclassified information) and specific cybersecurity requirements in federal programs for cloud service providers and defense contractors.

Many agencies, including the Departments of Defense and Homeland Security, also have their own AI standards. Other issues that may arise include requirements to conduct Privacy Impact Assessments (to demonstrate that AI systems respect civil rights, particularly in a law enforcement or surveillance context), as well as export-control and national security-related requirements.

How E&C Can Support Business in Getting Ahead of Risks

In order to meaningfully apply this patchwork of standards and best practices to mitigate AI risks, ethics and compliance professionals need to ask a number of questions in order to be effective in this role. While this list of questions must be tailored to a given organization and its risk profile and updated to reflect the evolving technology and regulatory framework, here are some key questions to consider:

How Is Your Business Developing or Deploying AI:

Are you developing the AI system or deploying a third-party system?
- If developing:
  - Are you training the AI system?
    - Do you have appropriate consent or another lawful basis for obtaining and using the data?
    - Is the data sufficiently representative?
  - Have you followed responsible AI principles?
- Are appropriate AI system guardrails in place?
- Have you complied with relevant AI regulations?
Who are the customers who will purchase, or on whose behalf you will use the AI solution?
- If the federal government, or if you will be a subcontractor on federal contracts, have you considered applicable regulations and flow-down requirements?
  - Are there any agency-specific rules?
- Do contracts with prospective customers have relevant requirements regarding system development, access, etc.?
In what jurisdictions will you be using or selling the system?
- What laws or rules apply (AI-specific laws, other relevant laws such as GDPR, industry-specific regulations, e.g., financial regulations, HIPAA)?

Risk Assessment & Impact Analysis

Is this a high-risk use case under applicable laws (i.e., the EU AI Act)?
- If yes, do you have a plan to comply?
Does the system make or inform consequential decisions about people?
- If yes, is there meaningful human review?
- Are processes sufficiently transparent for audit and legal defense purposes?
Could the system impact human rights?
- If so, what are foreseeable potential harms to individuals or communities, and have mitigation strategies been identified?

Model Integrity

Has the system been tested for bias, robustness, and explainability?
Are safeguards in place to prevent data leakage or adversarial manipulation?

Security and Technical Controls

Have you assessed cybersecurity threats such as model poisoning or prompt injection?
Is there a plan for continuous monitoring and incident response?

Employee Training

Have employees been trained on the risks and limitations of the system?

Key Components of an AI Compliance Program

Once you understand how the business is developing or deploying the AI system and the related risks and issues, you will be well-positioned to establish an appropriately tailored and risk-based approach. Many, if not all, of the AI-related risks you identify can be addressed through enhancements to your existing compliance programs (e.g., data privacy, confidentiality, information security, etc.) Below are AI-specific compliance program components to consider:

Governance and Oversight

Establish a cross-functional AI committee with business, technical, security, legal, and compliance representation.
Assign accountability at senior leadership and board levels.
Develop a governance framework (leveraging the questions above) to apply to new AI products.
- Frameworks should cover development, deployment, contracting, and use.
- Define approval requirements and escalation protocols.
- Governance should apply to systems built internally and those procured from third parties.
- Gather input from technical and business stakeholders, not just legal and compliance.

Policies and Standards

Wherever possible, incorporate AI risks and compliance requirements into existing policies and programs.
Adopt a responsible AI policy covering issues such as bias testing, data quality, explainability, vendor obligations, and prohibited use cases.

Risk Assessment and Impact Analysis

Implement a robust risk assessment process for high-risk use cases (e.g., employment, credit, finance, healthcare, surveillance).
Develop plans to mitigate identified risks.

Processes and Controls

Implement formal data classification protocols (e.g., public, internal, restricted, confidential) that clearly identify what data may and may not be used in AI workflows.
Adopt appropriate technical and operational controls and safeguards to protect data and AI systems, including strong access controls.
Implement a process for contracts to incorporate explicit confidentiality obligations into AI vendor agreements—for example, a prohibition against training AI models on a company’s data without express written permission.

Training

Train employees on acceptable use of AI and approved AI tools, as well as the risks, limitations, and responsibilities of AI use.
- Educate employees about the risks of shadow AI use.
- Ensure that safe, approved AI tools are available to prevent the use of shadow AI.

Monitoring, Auditing, and Testing

Continuously monitor AI systems for drift, bias, and unintended outcomes.
Use red teams to simulate adversarial misuse.
Conduct threat monitoring for AI-specific cyber risks.
Conduct internal and external audits when appropriate.
Consider technical monitoring controls and detection mechanisms for shadow use, for example:
- Network and endpoint monitoring
- Data loss prevention tools
- Usage analytics to identify large uploads to generative AI endpoints.

Reporting and Incident Management

Publicize channels to report concerns.
Ensure issues are appropriately investigated and addressed.
Adopt a plan for how detected shadow use will be handled.

Documentation

Maintain comprehensive records around:
- System design
- Data sources
- Risk assessments and testing results
- Governance processes and decisions.

Conclusion

While AI is capable of having tremendous positive impacts, it is nonetheless a significant governance challenge. By asking the right questions and embedding AI risk considerations into existing compliance architecture, companies can effectively manage risks without hindering innovation. The companies that will be most successful are those that view AI governance not only as a defensive exercise, but as an opportunity to build trust, demonstrate integrity, and be true leaders in a rapidly evolving world.

Stacey Sprenkel

PARTNER

Head of Compliance, Ethics, Risk & Governance

Head of ESG & Sustainability