Security & Quality Assurance Tools for Duke's AI Models

Description

As Duke deploys more AI models locally for privacy & cost reasons, two challenges emerge: 1) ensuring these models are safe to use and 2) determining which ones actually work best for our needs. You'll tackle both problems by building infrastructure tools that Duke's IT teams can use. You'll create a security scanning framework that automatically evaluates AI models downloaded from public repositories like Hugging Face: checking for malicious code hidden in model files, compromised dependencies, and supply-chain vulnerabilities before they ever touch Duke's systems. Second, you'll build an evaluation framework that rigorously tests how well different models perform across various tasks (from IT support to creative writing) and different response styles (precise vs. imaginative), giving Duke a systematic way to choose the right model for each use case. 

These aren't theoretical exercises; you'll be creating pragmatic tools with dashboards, automated scanning pipelines, risk scoring rubrics, and comparative analytics that could be deployed across Duke's infrastructure. You'll gain deep experience with AI model architectures, security vulnerability analysis, cloud and on-premises systems, & building robust evaluation frameworks. These skills position you at the forefront of the rapidly growing field of AI operations and security.