# Ensure Amazon SageMaker training jobs have VPC settings configured - Source: https://support.icompaas.com/support/solutions/articles/a_1793292225329785-ensure-amazon-sagemaker-training-jobs-have-vpc-settings-configured - Section: [Solutions](https://support.icompaas.com/support/solutions) › [AWS_Checks_v5.39.1](https://support.icompaas.com/support/solutions/c_1793285913477877) › [AWS Checks](https://support.icompaas.com/support/solutions/folders/f_1793287194371224) - Last updated: 2026-10-29 - Publisher: iCompaas, Inc. (https://www.icompaas.com) **Description** Ensure that every Amazon SageMaker training job has VPC settings configured, with at least one subnet and security group in its VpcConfig. With VPC settings, SageMaker places network interfaces for the training job in your VPC, so the traffic of the training containers is controlled by your security groups, route tables and VPC endpoints. **Rationale** Without VPC settings, training containers rely on public networking with broad outbound access. This makes it easier to exfiltrate datasets or model artifacts, download malicious code, and bypass security group controls and VPC-based monitoring. Running training jobs in specific subnets with tightly scoped security groups, VPC endpoints for Amazon S3, Amazon ECR and SageMaker, and outbound internet blocked by default keeps training traffic private. **Impact** - Places training traffic under your VPC security groups, routing and monitoring - Reduces the risk of data exfiltration from training containers - Existing training jobs cannot be changed, so the setting applies only to training jobs created after the change; existing failing jobs keep failing and must be accepted as exceptions - The subnets must provide access to Amazon S3 and Amazon ECR (through VPC endpoints or a NAT gateway), or the training job will fail to load its data and container image **Default Value** A training job has no VPC configuration unless subnets and security groups are specified when it is created. **Pre-Requisites** - IAM permissions: sagemaker:ListTrainingJobs, sagemaker:DescribeTrainingJob, sagemaker:CreateTrainingJob and iam:PassRole - A VPC with private subnets and a security group for the training containers - VPC endpoints (or a NAT gateway) so that the training containers can reach Amazon S3 and Amazon ECR - The execution role must allow the network interface actions SageMaker needs, such as ec2:CreateNetworkInterface and ec2:DescribeSubnets **Remediation** **Test Plan** **Console:** 1. Sign in to the AWS Management Console at [https://console.aws.amazon.com](https://console.aws.amazon.com/). 2. Open the Amazon SageMaker AI console and select **Training jobs** under **Training**. 3. Select a training job and review its network settings. 4. Check that a VPC with at least one subnet is configured. 5. Repeat for each training job in each region in use. 6. If any training job has no VPC subnets, follow the implementation steps. **CLI:** List the training jobs in a region: aws sagemaker list-training-jobs --region --query "TrainingJobSummaries\[\].\[TrainingJobName,TrainingJobStatus\]" --output table Show the VPC settings of a training job: aws sagemaker describe-training-job --region --training-job-name --query "VpcConfig" A training job fails if the output is null or contains no Subnets. All listed training jobs are evaluated, whatever their status. **Implementation Steps** Existing training jobs cannot be modified, and every training job that SageMaker still lists is evaluated. The original failing job therefore keeps failing even after a compliant clone is created. These steps prevent new failures; existing failing jobs must be accepted as exceptions, and they stop being reported only when SageMaker no longer returns them. **Console:** 1. Sign in to the AWS Management Console at [https://console.aws.amazon.com](https://console.aws.amazon.com/). 2. Open the Amazon SageMaker AI console and select **Training jobs** under **Training**. 3. Select the failing training job, choose **Actions**, then **Clone** (or choose **Create training job**). 4. In the network section, select a VPC, then choose at least one subnet and one security group. 5. Complete the remaining settings and choose **Create training job**. 6. Update the pipelines, notebooks or scripts that create training jobs so that new jobs include VPC settings: - in the CreateTrainingJob API, set VpcConfig with Subnets and SecurityGroupIds, - in the SageMaker Python SDK, set subnets and security\_group\_ids on the estimator. 7. Repeat for each affected training job in each region in use. **Backout Plan** 1. Sign in to the AWS Management Console at [https://console.aws.amazon.com](https://console.aws.amazon.com/). 2. Open the Amazon SageMaker AI console and select **Training jobs** under **Training**. 3. Create new training jobs without VPC settings, or remove the VPC settings from the pipelines, notebooks or scripts that create them. Note: rolling back removes the network isolation of the training containers. **Reference** - [https://docs.aws.amazon.com/sagemaker/latest/dg/train-vpc.html](https://docs.aws.amazon.com/sagemaker/latest/dg/train-vpc.html) - [https://docs.aws.amazon.com/sagemaker/latest/dg/interface-vpc-endpoint.html](https://docs.aws.amazon.com/sagemaker/latest/dg/interface-vpc-endpoint.html) - [https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-training-job.html](https://docs.aws.amazon.com/cli/latest/reference/sagemaker/describe-training-job.html) ## Related compliance guidance - [SOC 2 compliance: a practical implementation guide for cloud-native SaaS teams](https://support.icompaas.com/compliance/soc-2-trust-services-criteria): Configuration checks like this one are the technical half of a SOC 2 / ISO 27001 programme — the guide shows where they fit, what else an auditor expects, and how to evidence them. - [ISO/IEC 27001:2022 compliance: a practical implementation guide](https://support.icompaas.com/compliance/iso-27001-2022): ISO 27001 Annex A.8 (technological controls) is where cloud hardening evidence lands; the guide shows how to run the ISMS around it.