[AWS] SageMaker에서 S3 버킷에 저장된 학습 데이터 사용하기
이전 포스트에서는 AWS에서 자체 제공하는 기계 학습 플랫폼인 SageMaker를 사용하여 기계 학습 환경을 구축해봤는데요.
SageMaker에서 기계 학습 시 로컬 디렉토리나 다운로드 받은 기계 학습 데이터를 사용하는 것이 아닌 S3 버킷에 저장된 기계 학습 데이터를 사용하여 기계 학습을 실행할 수 있도록 해보겠습니다.
이전이 작성한 포스트인 Amazon SageMaker를 사용하여 기계 학습 환경을 구축하는 방법을 기준으로 진행하는 작업이므로 이전 포스트를 사전에 확인 후 작업하시면 더 도움이 될 것 같습니다.
SageMaker S3 버킷 확인
SageMaker Studio에서 Jupyter Notebook을 통해 Session 생성 시 Default S3 버킷이 생성됩니다.
아래 명령어를 통해 Default S3 버킷 이름을 확인하실 수 있습니다.
import sagemaker
#========== 기본설정 ==========
# Session 생성
sagemaker_session = sagemaker.Session()
# Amazon S3 bucket 이름
s3_bucket_name = sagemaker_session.default_bucket()
# Amazon S3 bucket 출력
print (s3_bucket_name)
기계 학습 데이터 S3 버킷 업로드
기계 학습 데이터는 COCO 데이터 셋을 사용하도록 하겠습니다.
SageMaker Default S3 버킷에 폴더를 생성 후 COCO 데이터 셋을 업로드 합니다.
업로드 작업은 AWS Console에서 작업하거나 AWS CLI를 통해 S3 버킷에 기계 학습 데이터를 업로드 할 수 있습니다.
AWS CLI 예시
# aws s3 cp {{기계 학습 데이터}} s3://{{S3 버킷 URL}}/coco128/images/.
기계 학습 코드
기계 학습 코드 - 기본 설정
SageMaker Default S3 버킷을 확인하고 COCO 데이터 셋 경로와 모델 저장 경로를 설정합니다.
import sagemaker
#========== 기본설정 ==========
# Session 생성
sagemaker_session = sagemaker.Session()
# Amazon S3 bucket 이름
s3_bucket_name = sagemaker_session.default_bucket()
# IAM role
role = sagemaker.get_execution_role()
#========== 학습 설정 ==========
# 데이터셋 경로
dataset_path = 'coco128'
# 모델 저장 경로
output_path = 'coco128/train'
# S3 bucket 데이터셋 경로
s3_dataset_path = f's3://{s3_bucket_name}/{dataset_path}'
# S3 bucket 모델 저장 경로
s3_output_path = f's3://{s3_bucket_name}/{output_path}'
# Estimator 설정
est_pytorch_entry_point= 'train.py'
est_pytorch_image_uri = '{ECR_URL}/sagemaker:pytorch-training-2.0.0-gpu-py310-cu118-ubuntu20.04-sagemaker'
est_pytorch_framework_version = '2.0.0'
est_pytorch_py_version = 'py310'
est_pytorch_instance_type = 'ml.g4dn.xlarge'
est_pytorch_instance_count = 1
est_pytorch_source_dir = './'
# Hyperparameters 설정
est_pytorch_hyperparameters={
'data': 'coco128.yaml',
'cfg': 'yolov5s.yaml',
'epochs': 3,
'project' : '/opt/ml/model',
'batch-size': 8
}
- SageMaker Default S3 버킷 이름을 설정합니다.
s3_bucket_name = sagemaker_session.default_bucket()
- S3 버킷관련 경로 설정을 추가합니다.
- 데이터셋 경로 :
dataset_path = 'coco128' - 모델 저장 경로 :
output_path = 'coco128/train' - S3 bucket 데이터셋 경로 :
s3_dataset_path = f's3://{s3_bucket_name}/{dataset_path}' - S3 bucket 모델 저장 경로 :
s3_output_path = f's3://{s3_bucket_name}/{output_path}'
- 데이터셋 경로 :
기계 학습 코드 - 입력 데이터 채널 및 Estimator 생성
기계 학습 입력 데이터 채널을 생성하고, 모델 저장 경로를 Estimator 생성 시 설정합니다.
from sagemaker.pytorch import PyTorch
from sagemaker.inputs import TrainingInput
# 입력 데이터 채널 (기계 학습 데이터)
data_inputs = TrainingInput(s3_data=s3_dataset_path, content_type='application/x-image')
# Estimator 생성
pytorch_estimator = PyTorch(
entry_point=est_pytorch_entry_point,
image_uri=est_pytorch_image_uri,
framework_version=est_pytorch_framework_version,
py_version=est_pytorch_py_version,
instance_type=est_pytorch_instance_type,
instance_count=est_pytorch_instance_count,
source_dir=est_pytorch_source_dir,
output_path=s3_output_path,
train_output=s3_output_path,
sagemaker_session=sagemaker_session,
role=role,
hyperparameters=est_pytorch_hyperparameters
)
- 사전에 S3 버킷에 업로드한 기계 학습 데이터를 사용할 수 있도록 기계 학습 입력 데이터 채널을 생성합니다.
- Estimator 생성 시
output_path,train_output옵션을 추가하여 모델 저장 경로를 설정합니다.
기계 학습 코드 - 학습 시작
사전에 생성한 기계 학습 입력 데이터 채널을 기계 학습 시작 시 사용하는 .fit() 함수에 추가합니다.
# 학습 시작
pytorch_estimator.fit(inputs=data_inputs)
기계 학습 코드 - 데이터 셋 경로 설정 변경
S3 버킷을 통해 기계 학습 데이터 다운로드 시 기본 경로인 /opt/ml/input/data 경로에 데이터가 다운로드 됩니다.
coco128.yaml 파일을 사용하여 데이터 셋을 정의하고 있으며 해당 파일의 경로 설정을 변경하여 S3 버킷을 통해 다운로드 받은 기계 학습 데이터를 사용할 수 있도록 수정합니다.
# 기존
path: ../datasets/coco128 # dataset root dir
# 수정
path: /opt/ml/input/data/training
기계 학습 코드 실행
작성한 기계 학습 코드를 실행해보도록 하겠습니다.
Estimator의 fit 함수를 실행합니다.
기존에는 학습 데이터를 저장하고 있지 않았기 때문에 기계 학습 데이터를 다운로드 받으며,
S3 버킷의 기계 학습 데이터를 사용할 경우 데이터 셋을 확인하여 이후 작업을 진행합니다.
- 기존
- S3 버킷 기계 학습 데이터 사용
S3 버킷의 기계 학습 데이터를 사용하기 때문에 기계 학습 시간도 단축됩니다.
- 기존 (427)
- S3 버킷 기계 학습 데이터 사용 (363초)
- 기계 학습 완료 시 사전에 설정한 모델 저장 경로에 모델 파일이 저장되어 있는지 저장 경로를 확인할 수 있습니다.
- S3 버킷에 직접 접속 후 기계 학습이 완료된 모델 파일을 확인하시기 바랍니다.
전체 학습 상세 로그는 아래 버튼을 클릭하여 확인하시기 바랍니다.
학습 상세 로그
Using provided s3_resource
INFO:sagemaker:Creating training-job with name: sagemaker-2023-07-04-07-46-10-310
2023-07-04 07:46:13 Starting - Starting the training job...
2023-07-04 07:46:28 Starting - Preparing the instances for training......
2023-07-04 07:47:31 Downloading - Downloading input data...
2023-07-04 07:48:01 Training - Downloading the training image...........................
2023-07-04 07:52:27 Training - Training image download completed. Training in progress.bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
2023-07-04 07:52:37,314 sagemaker-training-toolkit INFO Imported framework sagemaker_pytorch_container.training
2023-07-04 07:52:37,331 sagemaker-training-toolkit INFO No Neurons detected (normal if no neurons installed)
2023-07-04 07:52:37,340 sagemaker_pytorch_container.training INFO Block until all host DNS lookups succeed.
2023-07-04 07:52:37,345 sagemaker_pytorch_container.training INFO Invoking user training script.
2023-07-04 07:52:39,710 sagemaker-training-toolkit INFO Installing dependencies from requirements.txt:
/opt/conda/bin/python3.10 -m pip install -r requirements.txt
Collecting gitpython>=3.1.30 (from -r requirements.txt (line 5))
Downloading GitPython-3.1.31-py3-none-any.whl (184 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 184.3/184.3 kB 12.4 MB/s eta 0:00:00
Requirement already satisfied: matplotlib>=3.3 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 6)) (3.7.1)
Requirement already satisfied: numpy>=1.18.5 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 7)) (1.23.5)
Requirement already satisfied: opencv-python>=4.1.1 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 8)) (4.7.0)
Requirement already satisfied: Pillow>=7.1.2 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 9)) (9.4.0)
Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 10)) (5.9.5)
Requirement already satisfied: PyYAML>=5.3.1 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 11)) (5.4.1)
Requirement already satisfied: requests>=2.23.0 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 12)) (2.28.2)
Requirement already satisfied: scipy>=1.4.1 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 13)) (1.10.1)
Collecting thop>=0.1.1 (from -r requirements.txt (line 14))
Downloading thop-0.1.1.post2209072238-py3-none-any.whl (15 kB)
Requirement already satisfied: torch>=1.7.0 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 15)) (2.0.0)
Requirement already satisfied: torchvision>=0.8.1 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 16)) (0.15.1)
Requirement already satisfied: tqdm>=4.64.0 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 17)) (4.65.0)
Collecting ultralytics>=8.0.111 (from -r requirements.txt (line 18))
Downloading ultralytics-8.0.125-py3-none-any.whl (612 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 612.5/612.5 kB 80.2 MB/s eta 0:00:00
Requirement already satisfied: pandas>=1.1.4 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 27)) (2.0.1)
Requirement already satisfied: seaborn>=0.11.0 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 28)) (0.12.2)
Requirement already satisfied: setuptools>=65.5.1 in /opt/conda/lib/python3.10/site-packages (from -r requirements.txt (line 42)) (65.6.3)
Collecting gitdb<5,>=4.0.1 (from gitpython>=3.1.30->-r requirements.txt (line 5))
Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.7/62.7 kB 21.5 MB/s eta 0:00:00
Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (1.0.7)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (4.39.4)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (23.1)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.10/site-packages (from matplotlib>=3.3->-r requirements.txt (line 6)) (2.8.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests>=2.23.0->-r requirements.txt (line 12)) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests>=2.23.0->-r requirements.txt (line 12)) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests>=2.23.0->-r requirements.txt (line 12)) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests>=2.23.0->-r requirements.txt (line 12)) (2023.5.7)
Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from torch>=1.7.0->-r requirements.txt (line 15)) (3.12.0)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from torch>=1.7.0->-r requirements.txt (line 15)) (4.5.0)
Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.7.0->-r requirements.txt (line 15)) (1.11.1)
Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.7.0->-r requirements.txt (line 15)) (3.1)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.7.0->-r requirements.txt (line 15)) (3.1.2)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas>=1.1.4->-r requirements.txt (line 27)) (2023.3)
Requirement already satisfied: tzdata>=2022.1 in /opt/conda/lib/python3.10/site-packages (from pandas>=1.1.4->-r requirements.txt (line 27)) (2023.3)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython>=3.1.30->-r requirements.txt (line 5))
Downloading smmap-5.0.0-py3-none-any.whl (24 kB)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib>=3.3->-r requirements.txt (line 6)) (1.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.7.0->-r requirements.txt (line 15)) (2.1.2)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.7.0->-r requirements.txt (line 15)) (1.3.0)
Installing collected packages: smmap, gitdb, thop, gitpython, ultralytics
Successfully installed gitdb-4.0.10 gitpython-3.1.31 smmap-5.0.0 thop-0.1.1.post2209072238 ultralytics-8.0.125
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
2023-07-04 07:52:42,410 sagemaker-training-toolkit INFO Waiting for the process to finish and give a return code.
2023-07-04 07:52:42,411 sagemaker-training-toolkit INFO Done waiting for a return code. Received 0 from exiting process.
2023-07-04 07:52:42,428 sagemaker-training-toolkit INFO No Neurons detected (normal if no neurons installed)
2023-07-04 07:52:42,453 sagemaker-training-toolkit INFO No Neurons detected (normal if no neurons installed)
2023-07-04 07:52:42,477 sagemaker-training-toolkit INFO No Neurons detected (normal if no neurons installed)
2023-07-04 07:52:42,485 sagemaker-training-toolkit INFO Invoking user script
Training Env:
{
"additional_framework_parameters": {},
"channel_input_dirs": {
"training": "/opt/ml/input/data/training"
},
"current_host": "algo-1",
"current_instance_group": "homogeneousCluster",
"current_instance_group_hosts": [
"algo-1"
],
"current_instance_type": "ml.g4dn.xlarge",
"distribution_hosts": [],
"distribution_instance_groups": [],
"framework_module": "sagemaker_pytorch_container.training:main",
"hosts": [
"algo-1"
],
"hyperparameters": {
"batch-size": 8,
"cfg": "yolov5s.yaml",
"data": "coco128.yaml",
"epochs": 3,
"project": "/opt/ml/model/"
},
"input_config_dir": "/opt/ml/input/config",
"input_data_config": {
"training": {
"ContentType": "application/x-image",
"TrainingInputMode": "File",
"S3DistributionType": "FullyReplicated",
"RecordWrapperType": "None"
}
},
"input_dir": "/opt/ml/input",
"instance_groups": [
"homogeneousCluster"
],
"instance_groups_dict": {
"homogeneousCluster": {
"instance_group_name": "homogeneousCluster",
"instance_type": "ml.g4dn.xlarge",
"hosts": [
"algo-1"
]
}
},
"is_hetero": false,
"is_master": true,
"is_modelparallel_enabled": null,
"is_smddpmprun_installed": true,
"job_name": "sagemaker-2023-07-04-07-46-10-310",
"log_level": 20,
"master_hostname": "algo-1",
"model_dir": "/opt/ml/model",
"module_dir": "s3://####################/sagemaker-2023-07-04-07-46-10-310/source/sourcedir.tar.gz",
"module_name": "train",
"network_interface_name": "eth0",
"num_cpus": 4,
"num_gpus": 1,
"num_neurons": 0,
"output_data_dir": "/opt/ml/output/data",
"output_dir": "/opt/ml/output",
"output_intermediate_dir": "/opt/ml/output/intermediate",
"resource_config": {
"current_host": "algo-1",
"current_instance_type": "ml.g4dn.xlarge",
"current_group_name": "homogeneousCluster",
"hosts": [
"algo-1"
],
"instance_groups": [
{
"instance_group_name": "homogeneousCluster",
"instance_type": "ml.g4dn.xlarge",
"hosts": [
"algo-1"
]
}
],
"network_interface_name": "eth0"
},
"user_entry_point": "train.py"
}
Environment variables:
SM_HOSTS=["algo-1"]
SM_NETWORK_INTERFACE_NAME=eth0
SM_HPS={"batch-size":8,"cfg":"yolov5s.yaml","data":"coco128.yaml","epochs":3,"project":"/opt/ml/model/"}
SM_USER_ENTRY_POINT=train.py
SM_FRAMEWORK_PARAMS={}
SM_RESOURCE_CONFIG={"current_group_name":"homogeneousCluster","current_host":"algo-1","current_instance_type":"ml.g4dn.xlarge","hosts":["algo-1"],"instance_groups":[{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.g4dn.xlarge"}],"network_interface_name":"eth0"}
SM_INPUT_DATA_CONFIG={"training":{"ContentType":"application/x-image","RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"}}
SM_OUTPUT_DATA_DIR=/opt/ml/output/data
SM_CHANNELS=["training"]
SM_CURRENT_HOST=algo-1
SM_CURRENT_INSTANCE_TYPE=ml.g4dn.xlarge
SM_CURRENT_INSTANCE_GROUP=homogeneousCluster
SM_CURRENT_INSTANCE_GROUP_HOSTS=["algo-1"]
SM_INSTANCE_GROUPS=["homogeneousCluster"]
SM_INSTANCE_GROUPS_DICT={"homogeneousCluster":{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.g4dn.xlarge"}}
SM_DISTRIBUTION_INSTANCE_GROUPS=[]
SM_IS_HETERO=false
SM_MODULE_NAME=train
SM_LOG_LEVEL=20
SM_FRAMEWORK_MODULE=sagemaker_pytorch_container.training:main
SM_INPUT_DIR=/opt/ml/input
SM_INPUT_CONFIG_DIR=/opt/ml/input/config
SM_OUTPUT_DIR=/opt/ml/output
SM_NUM_CPUS=4
SM_NUM_GPUS=1
SM_NUM_NEURONS=0
SM_MODEL_DIR=/opt/ml/model
SM_MODULE_DIR=s3://####################/sagemaker-2023-07-04-07-46-10-310/source/sourcedir.tar.gz
SM_TRAINING_ENV={"additional_framework_parameters":{},"channel_input_dirs":{"training":"/opt/ml/input/data/training"},"current_host":"algo-1","current_instance_group":"homogeneousCluster","current_instance_group_hosts":["algo-1"],"current_instance_type":"ml.g4dn.xlarge","distribution_hosts":[],"distribution_instance_groups":[],"framework_module":"sagemaker_pytorch_container.training:main","hosts":["algo-1"],"hyperparameters":{"batch-size":8,"cfg":"yolov5s.yaml","data":"coco128.yaml","epochs":3,"project":"/opt/ml/model/"},"input_config_dir":"/opt/ml/input/config","input_data_config":{"training":{"ContentType":"application/x-image","RecordWrapperType":"None","S3DistributionType":"FullyReplicated","TrainingInputMode":"File"}},"input_dir":"/opt/ml/input","instance_groups":["homogeneousCluster"],"instance_groups_dict":{"homogeneousCluster":{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.g4dn.xlarge"}},"is_hetero":false,"is_master":true,"is_modelparallel_enabled":null,"is_smddpmprun_installed":true,"job_name":"sagemaker-2023-07-04-07-46-10-310","log_level":20,"master_hostname":"algo-1","model_dir":"/opt/ml/model","module_dir":"s3://####################/sagemaker-2023-07-04-07-46-10-310/source/sourcedir.tar.gz","module_name":"train","network_interface_name":"eth0","num_cpus":4,"num_gpus":1,"num_neurons":0,"output_data_dir":"/opt/ml/output/data","output_dir":"/opt/ml/output","output_intermediate_dir":"/opt/ml/output/intermediate","resource_config":{"current_group_name":"homogeneousCluster","current_host":"algo-1","current_instance_type":"ml.g4dn.xlarge","hosts":["algo-1"],"instance_groups":[{"hosts":["algo-1"],"instance_group_name":"homogeneousCluster","instance_type":"ml.g4dn.xlarge"}],"network_interface_name":"eth0"},"user_entry_point":"train.py"}
SM_USER_ARGS=["--batch-size","8","--cfg","yolov5s.yaml","--data","coco128.yaml","--epochs","3","--project","/opt/ml/model/"]
SM_OUTPUT_INTERMEDIATE_DIR=/opt/ml/output/intermediate
SM_CHANNEL_TRAINING=/opt/ml/input/data/training
SM_HP_BATCH-SIZE=8
SM_HP_CFG=yolov5s.yaml
SM_HP_DATA=coco128.yaml
SM_HP_EPOCHS=3
SM_HP_PROJECT=/opt/ml/model/
PYTHONPATH=/opt/ml/code:/opt/conda/bin:/opt/conda/lib/python310.zip:/opt/conda/lib/python3.10:/opt/conda/lib/python3.10/lib-dynload:/opt/conda/lib/python3.10/site-packages
Invoking script with the following command:
/opt/conda/bin/python3.10 train.py --batch-size 8 --cfg yolov5s.yaml --data coco128.yaml --epochs 3 --project /opt/ml/model/
2023-07-04 07:52:42,517 sagemaker-training-toolkit INFO Exceptions not imported for SageMaker TF as Tensorflow is not installed.
#033[34m#033[1mtrain: #033[0mweights=yolov5s.pt, cfg=yolov5s.yaml, data=coco128.yaml, hyp=data/hyps/hyp.scratch-low.yaml, epochs=3, batch_size=8, imgsz=640, rect=False, resume=False, nosave=False, noval=False, noautoanchor=False, noplots=False, evolve=None, bucket=, cache=None, image_weights=False, device=, multi_scale=False, single_cls=False, optimizer=SGD, sync_bn=False, workers=8, project=/opt/ml/model/, name=exp, exist_ok=False, quad=False, cos_lr=False, label_smoothing=0.0, patience=100, freeze=[0], save_period=-1, seed=0, local_rank=-1, entity=None, upload_dataset=False, bbox_interval=-1, artifact_alias=latest
From https://github.com/ultralytics/yolov5
0004c74..a453a45 master -> origin/master
* [new branch] discord-link -> origin/discord-link
* [new branch] pre-commit-ci-update-config -> origin/pre-commit-ci-update-config
#033[34m#033[1mgithub: #033[0m⚠️ YOLOv5 is out of date by 2 commits. Use 'git pull' or 'git clone https://github.com/ultralytics/yolov5' to update.
YOLOv5 🚀 v7.0-187-g0004c74 Python-3.10.8 torch-2.0.0 CUDA:0 (Tesla T4, 15102MiB)
#033[34m#033[1mhyperparameters: #033[0mlr0=0.01, lrf=0.01, momentum=0.937, weight_decay=0.0005, warmup_epochs=3.0, warmup_momentum=0.8, warmup_bias_lr=0.1, box=0.05, cls=0.5, cls_pw=1.0, obj=1.0, obj_pw=1.0, iou_t=0.2, anchor_t=4.0, fl_gamma=0.0, hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=0.0, translate=0.1, scale=0.5, shear=0.0, perspective=0.0, flipud=0.0, fliplr=0.5, mosaic=1.0, mixup=0.0, copy_paste=0.0
#033[34m#033[1mComet: #033[0mrun 'pip install comet_ml' to automatically track and visualize YOLOv5 🚀 runs in Comet
#033[34m#033[1mTensorBoard: #033[0mStart with 'tensorboard --logdir /opt/ml/model', view at http://localhost:6006/
Downloading https://ultralytics.com/assets/Arial.ttf to /root/.config/Ultralytics/Arial.ttf...
0%| | 0.00/755k [00:00<?, ?B/s]
100%|██████████| 755k/755k [00:00<00:00, 42.6MB/s]
Downloading https://github.com/ultralytics/yolov5/releases/download/v7.0/yolov5s.pt to yolov5s.pt...
0%| | 0.00/14.1M [00:00<?, ?B/s]
100%|██████████| 14.1M/14.1M [00:00<00:00, 411MB/s]
from n params module arguments
0 -1 1 3520 models.common.Conv [3, 32, 6, 2, 2]
1 -1 1 18560 models.common.Conv [32, 64, 3, 2]
2 -1 1 18816 models.common.C3 [64, 64, 1]
3 -1 1 73984 models.common.Conv [64, 128, 3, 2]
4 -1 2 115712 models.common.C3 [128, 128, 2]
5 -1 1 295424 models.common.Conv [128, 256, 3, 2]
6 -1 3 625152 models.common.C3 [256, 256, 3]
7 -1 1 1180672 models.common.Conv [256, 512, 3, 2]
8 -1 1 1182720 models.common.C3 [512, 512, 1]
9 -1 1 656896 models.common.SPPF [512, 512, 5]
10 -1 1 131584 models.common.Conv [512, 256, 1, 1]
11 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
12 [-1, 6] 1 0 models.common.Concat [1]
13 -1 1 361984 models.common.C3 [512, 256, 1, False]
14 -1 1 33024 models.common.Conv [256, 128, 1, 1]
15 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
16 [-1, 4] 1 0 models.common.Concat [1]
17 -1 1 90880 models.common.C3 [256, 128, 1, False]
18 -1 1 147712 models.common.Conv [128, 128, 3, 2]
19 [-1, 14] 1 0 models.common.Concat [1]
20 -1 1 296448 models.common.C3 [256, 256, 1, False]
21 -1 1 590336 models.common.Conv [256, 256, 3, 2]
22 [-1, 10] 1 0 models.common.Concat [1]
23 -1 1 1182720 models.common.C3 [512, 512, 1, False]
24 [17, 20, 23] 1 229245 models.yolo.Detect [80, [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], [128, 256, 512]]
YOLOv5s summary: 214 layers, 7235389 parameters, 7235389 gradients, 16.6 GFLOPs
Transferred 348/349 items from yolov5s.pt
#033[34m#033[1mAMP: #033[0mchecks passed ✅
#033[34m#033[1moptimizer:#033[0m SGD(lr=0.01) with parameter groups 57 weight(decay=0.0), 60 weight(decay=0.0005), 60 bias
#033[34m#033[1mtrain: #033[0mScanning /opt/ml/input/data/training/labels/train2017...: 0%| | 0/128 [00:00<?, ?it/s]
#033[34m#033[1mtrain: #033[0mScanning /opt/ml/input/data/training/labels/train2017... 126 images, 2 backgrounds, 0 corrupt: 100%|██████████| 128/128 [00:00<00:00, 2417.66it/s]
#033[34m#033[1mtrain: #033[0mNew cache created: /opt/ml/input/data/training/labels/train2017.cache
#033[34m#033[1mval: #033[0mScanning /opt/ml/input/data/training/labels/train2017.cache... 126 images, 2 backgrounds, 0 corrupt: 100%|██████████| 128/128 [00:00<?, ?it/s]
#033[34m#033[1mval: #033[0mScanning /opt/ml/input/data/training/labels/train2017.cache... 126 images, 2 backgrounds, 0 corrupt: 100%|██████████| 128/128 [00:00<?, ?it/s]
#033[34m#033[1mAutoAnchor: #033[0m4.27 anchors/target, 0.994 Best Possible Recall (BPR). Current anchors are a good fit to dataset ✅
Plotting labels to /opt/ml/model/exp/labels.jpg...
Image sizes 640 train, 640 val
Using 4 dataloader workers
Logging results to #033[1m/opt/ml/model/exp#033[0m
Starting training for 3 epochs...
Epoch GPU_mem box_loss obj_loss cls_loss Instances Size
0%| | 0/16 [00:00<?, ?it/s]
0/2 1.72G 0.04263 0.04785 0.02002 84 640: 0%| | 0/16 [00:00<?, ?it/s]
0/2 1.72G 0.04263 0.04785 0.02002 84 640: 6%|▋ | 1/16 [00:00<00:07, 2.00it/s]
0/2 1.86G 0.04137 0.0532 0.01919 77 640: 6%|▋ | 1/16 [00:00<00:07, 2.00it/s]
0/2 1.86G 0.04137 0.0532 0.01919 77 640: 12%|█▎ | 2/16 [00:00<00:04, 3.39it/s]
0/2 1.86G 0.04399 0.05919 0.01962 86 640: 12%|█▎ | 2/16 [00:00<00:04, 3.39it/s]
0/2 1.86G 0.04399 0.05919 0.01962 86 640: 19%|█▉ | 3/16 [00:00<00:03, 3.98it/s]
0/2 1.86G 0.0457 0.06293 0.01933 123 640: 19%|█▉ | 3/16 [00:01<00:03, 3.98it/s]#015 0/2 1.86G 0.0457 0.06293 0.01933 123 640: 25%|██▌ | 4/16 [00:01<00:03, 3.71it/s]
0/2 1.86G 0.04631 0.06611 0.01771 115 640: 25%|██▌ | 4/16 [00:01<00:03, 3.71it/s]
0/2 1.86G 0.04631 0.06611 0.01771 115 640: 31%|███▏ | 5/16 [00:01<00:02, 4.62it/s]
0/2 1.86G 0.04695 0.06591 0.01847 114 640: 31%|███▏ | 5/16 [00:01<00:02, 4.62it/s]#015 0/2 1.86G 0.04695 0.06591 0.01847 114 640: 38%|███▊ | 6/16 [00:01<00:01, 5.28it/s]
0/2 1.86G 0.04622 0.06352 0.01832 78 640: 38%|███▊ | 6/16 [00:01<00:01, 5.28it/s]#015 0/2 1.86G 0.04622 0.06352 0.01832 78 640: 44%|████▍ | 7/16 [00:01<00:01, 5.99it/s]
0/2 1.86G 0.04573 0.06442 0.018 118 640: 44%|████▍ | 7/16 [00:01<00:01, 5.99it/s]
0/2 1.86G 0.04573 0.06442 0.018 118 640: 50%|█████ | 8/16 [00:01<00:01, 6.63it/s]
0/2 1.86G 0.04537 0.06323 0.01781 76 640: 50%|█████ | 8/16 [00:01<00:01, 6.63it/s]
0/2 1.86G 0.04574 0.06244 0.01824 91 640: 50%|█████ | 8/16 [00:01<00:01, 6.63it/s]
0/2 1.86G 0.04574 0.06244 0.01824 91 640: 62%|██████▎ | 10/16 [00:01<00:00, 7.67it/s]
0/2 1.86G 0.04525 0.06201 0.01792 96 640: 62%|██████▎ | 10/16 [00:01<00:00, 7.67it/s]
0/2 1.86G 0.04525 0.06201 0.01792 96 640: 69%|██████▉ | 11/16 [00:01<00:00, 8.13it/s]
0/2 1.86G 0.04521 0.0621 0.01862 103 640: 69%|██████▉ | 11/16 [00:02<00:00, 8.13it/s]#015 0/2 1.86G 0.04521 0.0621 0.01862 103 640: 75%|███████▌ | 12/16 [00:02<00:00, 7.87it/s]
0/2 1.86G 0.04485 0.06149 0.0183 91 640: 75%|███████▌ | 12/16 [00:02<00:00, 7.87it/s]
0/2 1.86G 0.04485 0.06149 0.0183 91 640: 81%|████████▏ | 13/16 [00:02<00:00, 8.36it/s]
0/2 1.86G 0.04482 0.06218 0.01863 116 640: 81%|████████▏ | 13/16 [00:02<00:00, 8.36it/s]
0/2 1.86G 0.04482 0.06218 0.01863 116 640: 88%|████████▊ | 14/16 [00:02<00:00, 8.19it/s]
0/2 1.86G 0.04545 0.06109 0.01889 63 640: 88%|████████▊ | 14/16 [00:02<00:00, 8.19it/s]#015 0/2 1.86G 0.04545 0.06109 0.01889 63 640: 94%|█████████▍| 15/16 [00:02<00:00, 8.26it/s]
0/2 1.86G 0.04523 0.06067 0.01926 82 640: 94%|█████████▍| 15/16 [00:02<00:00, 8.26it/s]#015 0/2 1.86G 0.04523 0.06067 0.01926 82 640: 100%|██████████| 16/16 [00:02<00:00, 8.03it/s]#015 0/2 1.86G 0.04523 0.06067 0.01926 82 640: 100%|██████████| 16/16 [00:02<00:00, 6.20it/s]
Class Images Instances P R mAP50 mAP50-95: 0%| | 0/8 [00:00<?, ?it/s]
Class Images Instances P R mAP50 mAP50-95: 12%|█▎ | 1/8 [00:01<00:10, 1.44s/it]
Class Images Instances P R mAP50 mAP50-95: 25%|██▌ | 2/8 [00:01<00:04, 1.48it/s]
Class Images Instances P R mAP50 mAP50-95: 38%|███▊ | 3/8 [00:01<00:02, 2.10it/s]
Class Images Instances P R mAP50 mAP50-95: 50%|█████ | 4/8 [00:01<00:01, 2.94it/s]
Class Images Instances P R mAP50 mAP50-95: 62%|██████▎ | 5/8 [00:02<00:00, 3.70it/s]
Class Images Instances P R mAP50 mAP50-95: 75%|███████▌ | 6/8 [00:02<00:00, 3.92it/s]
Class Images Instances P R mAP50 mAP50-95: 88%|████████▊ | 7/8 [00:02<00:00, 4.14it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:02<00:00, 4.35it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:02<00:00, 2.92it/s]
all 128 929 0.732 0.631 0.718 0.477
Epoch GPU_mem box_loss obj_loss cls_loss Instances Size
0%| | 0/16 [00:00<?, ?it/s]
1/2 2.03G 0.05467 0.06597 0.02623 109 640: 0%| | 0/16 [00:00<?, ?it/s]
1/2 2.03G 0.04909 0.0572 0.0236 60 640: 0%| | 0/16 [00:00<?, ?it/s]
1/2 2.03G 0.04909 0.0572 0.0236 60 640: 12%|█▎ | 2/16 [00:00<00:01, 10.01it/s]
1/2 2.03G 0.04965 0.06586 0.02222 137 640: 12%|█▎ | 2/16 [00:00<00:01, 10.01it/s]
1/2 2.04G 0.04856 0.07264 0.02058 143 640: 12%|█▎ | 2/16 [00:00<00:01, 10.01it/s]
1/2 2.04G 0.04856 0.07264 0.02058 143 640: 25%|██▌ | 4/16 [00:00<00:01, 10.01it/s]
1/2 2.04G 0.04746 0.07102 0.01933 91 640: 25%|██▌ | 4/16 [00:00<00:01, 10.01it/s]
1/2 2.04G 0.04695 0.0708 0.01908 94 640: 25%|██▌ | 4/16 [00:00<00:01, 10.01it/s]
1/2 2.04G 0.04695 0.0708 0.01908 94 640: 38%|███▊ | 6/16 [00:00<00:01, 9.95it/s]
1/2 2.04G 0.04689 0.0724 0.01959 135 640: 38%|███▊ | 6/16 [00:00<00:01, 9.95it/s]
1/2 2.04G 0.04749 0.07476 0.01983 159 640: 38%|███▊ | 6/16 [00:00<00:01, 9.95it/s]
1/2 2.04G 0.04749 0.07476 0.01983 159 640: 50%|█████ | 8/16 [00:00<00:00, 10.21it/s]
1/2 2.04G 0.04698 0.07365 0.01919 97 640: 50%|█████ | 8/16 [00:00<00:00, 10.21it/s]
1/2 2.04G 0.0468 0.07334 0.01888 109 640: 50%|█████ | 8/16 [00:00<00:00, 10.21it/s]
1/2 2.04G 0.0468 0.07334 0.01888 109 640: 62%|██████▎ | 10/16 [00:00<00:00, 9.99it/s]
1/2 2.04G 0.04633 0.07145 0.019 88 640: 62%|██████▎ | 10/16 [00:01<00:00, 9.99it/s]
1/2 2.04G 0.04585 0.07 0.019 79 640: 62%|██████▎ | 10/16 [00:01<00:00, 9.99it/s]
1/2 2.04G 0.04585 0.07 0.019 79 640: 75%|███████▌ | 12/16 [00:01<00:00, 9.89it/s]
1/2 2.04G 0.04636 0.06949 0.01866 104 640: 75%|███████▌ | 12/16 [00:01<00:00, 9.89it/s]
1/2 2.04G 0.04635 0.06968 0.01835 120 640: 75%|███████▌ | 12/16 [00:01<00:00, 9.89it/s]
1/2 2.04G 0.04635 0.06968 0.01835 120 640: 88%|████████▊ | 14/16 [00:01<00:00, 10.15it/s]
1/2 2.04G 0.04637 0.06777 0.01839 76 640: 88%|████████▊ | 14/16 [00:01<00:00, 10.15it/s]
1/2 2.04G 0.04622 0.06747 0.01825 116 640: 88%|████████▊ | 14/16 [00:01<00:00, 10.15it/s]
1/2 2.04G 0.04622 0.06747 0.01825 116 640: 100%|██████████| 16/16 [00:01<00:00, 8.42it/s]#015 1/2 2.04G 0.04622 0.06747 0.01825 116 640: 100%|██████████| 16/16 [00:01<00:00, 9.33it/s]
Class Images Instances P R mAP50 mAP50-95: 0%| | 0/8 [00:00<?, ?it/s]
Class Images Instances P R mAP50 mAP50-95: 12%|█▎ | 1/8 [00:00<00:00, 7.77it/s]
Class Images Instances P R mAP50 mAP50-95: 25%|██▌ | 2/8 [00:00<00:00, 7.41it/s]
Class Images Instances P R mAP50 mAP50-95: 38%|███▊ | 3/8 [00:00<00:00, 7.31it/s]
Class Images Instances P R mAP50 mAP50-95: 50%|█████ | 4/8 [00:00<00:00, 7.30it/s]
Class Images Instances P R mAP50 mAP50-95: 62%|██████▎ | 5/8 [00:00<00:00, 7.08it/s]
Class Images Instances P R mAP50 mAP50-95: 75%|███████▌ | 6/8 [00:00<00:00, 6.67it/s]
Class Images Instances P R mAP50 mAP50-95: 88%|████████▊ | 7/8 [00:01<00:00, 6.77it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:01<00:00, 6.87it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:01<00:00, 6.99it/s]
all 128 929 0.785 0.629 0.738 0.498
Epoch GPU_mem box_loss obj_loss cls_loss Instances Size
0%| | 0/16 [00:00<?, ?it/s]
2/2 2.04G 0.04802 0.1227 0.01417 195 640: 0%| | 0/16 [00:00<?, ?it/s]
2/2 2.04G 0.04561 0.08766 0.01594 101 640: 0%| | 0/16 [00:00<?, ?it/s]
2/2 2.04G 0.04561 0.08766 0.01594 101 640: 12%|█▎ | 2/16 [00:00<00:01, 9.64it/s]
2/2 2.04G 0.04542 0.07617 0.01757 86 640: 12%|█▎ | 2/16 [00:00<00:01, 9.64it/s]
2/2 2.04G 0.04423 0.07265 0.01708 98 640: 12%|█▎ | 2/16 [00:00<00:01, 9.64it/s]
2/2 2.04G 0.04423 0.07265 0.01708 98 640: 25%|██▌ | 4/16 [00:00<00:01, 10.27it/s]
2/2 2.04G 0.04398 0.07078 0.01723 111 640: 25%|██▌ | 4/16 [00:00<00:01, 10.27it/s]
2/2 2.04G 0.04369 0.07246 0.01751 113 640: 25%|██▌ | 4/16 [00:00<00:01, 10.27it/s]
2/2 2.04G 0.04369 0.07246 0.01751 113 640: 38%|███▊ | 6/16 [00:00<00:01, 10.00it/s]
2/2 2.04G 0.04346 0.06956 0.018 102 640: 38%|███▊ | 6/16 [00:00<00:01, 10.00it/s]
2/2 2.04G 0.04342 0.06977 0.01773 118 640: 38%|███▊ | 6/16 [00:00<00:01, 10.00it/s]
2/2 2.04G 0.04342 0.06977 0.01773 118 640: 50%|█████ | 8/16 [00:00<00:00, 10.21it/s]
2/2 2.04G 0.04328 0.06999 0.01823 102 640: 50%|█████ | 8/16 [00:00<00:00, 10.21it/s]
2/2 2.04G 0.04326 0.06942 0.01802 113 640: 50%|█████ | 8/16 [00:00<00:00, 10.21it/s]
2/2 2.04G 0.04326 0.06942 0.01802 113 640: 62%|██████▎ | 10/16 [00:00<00:00, 10.05it/s]
2/2 2.04G 0.04364 0.06832 0.01798 118 640: 62%|██████▎ | 10/16 [00:01<00:00, 10.05it/s]
2/2 2.04G 0.04363 0.06963 0.01759 140 640: 62%|██████▎ | 10/16 [00:01<00:00, 10.05it/s]
2/2 2.04G 0.04363 0.06963 0.01759 140 640: 75%|███████▌ | 12/16 [00:01<00:00, 10.25it/s]
2/2 2.04G 0.04389 0.06915 0.0175 107 640: 75%|███████▌ | 12/16 [00:01<00:00, 10.25it/s]
2/2 2.04G 0.04385 0.0688 0.01785 98 640: 75%|███████▌ | 12/16 [00:01<00:00, 10.25it/s]
2/2 2.04G 0.04385 0.0688 0.01785 98 640: 88%|████████▊ | 14/16 [00:01<00:00, 10.07it/s]
2/2 2.04G 0.04383 0.06737 0.01852 62 640: 88%|████████▊ | 14/16 [00:01<00:00, 10.07it/s]
2/2 2.04G 0.04365 0.06745 0.01911 96 640: 88%|████████▊ | 14/16 [00:01<00:00, 10.07it/s]
2/2 2.04G 0.04365 0.06745 0.01911 96 640: 100%|██████████| 16/16 [00:01<00:00, 8.03it/s]#015 2/2 2.04G 0.04365 0.06745 0.01911 96 640: 100%|██████████| 16/16 [00:01<00:00, 9.17it/s]
Class Images Instances P R mAP50 mAP50-95: 0%| | 0/8 [00:00<?, ?it/s]
Class Images Instances P R mAP50 mAP50-95: 12%|█▎ | 1/8 [00:00<00:00, 7.89it/s]
Class Images Instances P R mAP50 mAP50-95: 25%|██▌ | 2/8 [00:00<00:00, 7.56it/s]
Class Images Instances P R mAP50 mAP50-95: 38%|███▊ | 3/8 [00:00<00:00, 7.36it/s]
Class Images Instances P R mAP50 mAP50-95: 50%|█████ | 4/8 [00:00<00:00, 7.44it/s]
Class Images Instances P R mAP50 mAP50-95: 62%|██████▎ | 5/8 [00:00<00:00, 6.98it/s]
Class Images Instances P R mAP50 mAP50-95: 75%|███████▌ | 6/8 [00:00<00:00, 6.80it/s]
Class Images Instances P R mAP50 mAP50-95: 88%|████████▊ | 7/8 [00:01<00:00, 6.67it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:01<00:00, 6.90it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:01<00:00, 7.03it/s]
all 128 929 0.819 0.627 0.747 0.504
3 epochs completed in 0.004 hours.
Optimizer stripped from /opt/ml/model/exp/weights/last.pt, 14.8MB
Optimizer stripped from /opt/ml/model/exp/weights/best.pt, 14.8MB
Validating /opt/ml/model/exp/weights/best.pt...
Fusing layers...
YOLOv5s summary: 157 layers, 7225885 parameters, 0 gradients, 16.4 GFLOPs
Class Images Instances P R mAP50 mAP50-95: 0%| | 0/8 [00:00<?, ?it/s]
Class Images Instances P R mAP50 mAP50-95: 12%|█▎ | 1/8 [00:00<00:01, 5.84it/s]
Class Images Instances P R mAP50 mAP50-95: 25%|██▌ | 2/8 [00:00<00:01, 3.41it/s]
Class Images Instances P R mAP50 mAP50-95: 38%|███▊ | 3/8 [00:00<00:01, 2.83it/s]
Class Images Instances P R mAP50 mAP50-95: 50%|█████ | 4/8 [00:01<00:01, 2.61it/s]
Class Images Instances P R mAP50 mAP50-95: 62%|██████▎ | 5/8 [00:01<00:00, 3.20it/s]
Class Images Instances P R mAP50 mAP50-95: 75%|███████▌ | 6/8 [00:01<00:00, 3.68it/s]
Class Images Instances P R mAP50 mAP50-95: 88%|████████▊ | 7/8 [00:01<00:00, 4.16it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:02<00:00, 4.61it/s]
Class Images Instances P R mAP50 mAP50-95: 100%|██████████| 8/8 [00:02<00:00, 3.76it/s]
all 128 929 0.819 0.627 0.746 0.503
person 128 254 0.9 0.669 0.807 0.523
bicycle 128 6 1 0.331 0.711 0.427
car 128 46 0.859 0.413 0.579 0.243
motorcycle 128 5 0.682 0.8 0.798 0.663
airplane 128 6 0.979 1 0.995 0.755
bus 128 7 0.748 0.714 0.779 0.659
train 128 3 1 0.659 0.863 0.532
truck 128 12 0.664 0.333 0.501 0.242
boat 128 6 1 0.301 0.519 0.252
traffic light 128 14 0.607 0.214 0.377 0.222
stop sign 128 2 0.792 1 0.995 0.772
bench 128 9 0.823 0.52 0.717 0.311
bird 128 16 0.885 1 0.991 0.651
cat 128 4 0.918 1 0.995 0.77
dog 128 9 1 0.653 0.901 0.693
horse 128 2 0.89 1 0.995 0.622
elephant 128 17 0.972 0.882 0.939 0.651
bear 128 1 0.71 1 0.995 0.995
zebra 128 4 0.885 1 0.995 0.93
giraffe 128 9 0.877 0.791 0.962 0.756
backpack 128 6 1 0.562 0.836 0.428
umbrella 128 18 0.79 0.629 0.877 0.529
handbag 128 19 0.959 0.158 0.337 0.188
tie 128 7 0.817 0.645 0.777 0.486
suitcase 128 4 0.866 1 0.995 0.601
frisbee 128 5 0.735 0.8 0.8 0.627
skis 128 1 0.827 1 0.995 0.497
snowboard 128 7 0.85 0.714 0.867 0.544
sports ball 128 6 0.685 0.667 0.668 0.346
kite 128 10 0.835 0.508 0.628 0.22
baseball bat 128 4 0.521 0.25 0.388 0.171
baseball glove 128 7 0.763 0.429 0.47 0.276
skateboard 128 5 0.725 0.54 0.648 0.479
tennis racket 128 7 0.801 0.429 0.559 0.344
bottle 128 18 0.588 0.333 0.601 0.31
wine glass 128 16 0.735 0.875 0.89 0.448
cup 128 36 0.902 0.667 0.839 0.52
fork 128 6 1 0.311 0.439 0.302
knife 128 16 0.829 0.625 0.704 0.393
spoon 128 22 0.828 0.439 0.637 0.363
bowl 128 28 1 0.63 0.763 0.573
banana 128 1 0.809 1 0.995 0.302
sandwich 128 2 1 0 0.448 0.383
orange 128 4 0.829 1 0.995 0.738
broccoli 128 11 0.824 0.364 0.491 0.356
carrot 128 24 0.685 0.625 0.713 0.487
hot dog 128 2 0.547 1 0.663 0.614
pizza 128 5 1 0.792 0.962 0.77
donut 128 14 0.674 1 0.946 0.799
cake 128 4 0.774 1 0.995 0.846
chair 128 35 0.612 0.632 0.619 0.319
couch 128 6 1 0.646 0.826 0.527
potted plant 128 14 0.903 0.786 0.875 0.508
bed 128 3 1 0 0.863 0.532
dining table 128 13 0.804 0.317 0.621 0.407
toilet 128 2 0.857 1 0.995 0.796
tv 128 2 0.751 1 0.995 0.796
laptop 128 3 1 0 0.913 0.548
mouse 128 2 1 0 0.0907 0.0454
remote 128 8 1 0.617 0.629 0.513
cell phone 128 8 0.722 0.332 0.453 0.262
microwave 128 3 0.848 1 0.995 0.843
oven 128 5 0.667 0.4 0.43 0.298
sink 128 6 0.272 0.167 0.338 0.252
refrigerator 128 5 0.672 0.8 0.81 0.558
book 128 29 0.749 0.206 0.367 0.168
clock 128 9 0.762 0.778 0.879 0.689
vase 128 2 0.439 1 0.995 0.895
scissors 128 1 1 0 0.166 0.0332
teddy bear 128 21 0.85 0.571 0.788 0.506
toothbrush 128 5 0.826 1 0.995 0.618
Results saved to #033[1m/opt/ml/model/exp#033[0m
2023-07-04 07:53:23,480 sagemaker-training-toolkit INFO Waiting for the process to finish and give a return code.
2023-07-04 07:53:23,480 sagemaker-training-toolkit INFO Done waiting for a return code. Received 0 from exiting process.
2023-07-04 07:53:23,480 sagemaker-training-toolkit INFO Reporting training SUCCESS
2023-07-04 07:53:38 Uploading - Uploading generated training model
2023-07-04 07:53:38 Completed - Training job completed
Training seconds: 368
Billable seconds: 368
SageMaker에서 S3 버킷에 저장된 기계 학습 데이터를 사용하는 작업을 완료하였습니다.
이제 기계 학습 시 사용하는 기계 학습 데이터를 S3 버킷에 저장 및 관리하면서 기계 학습을 사용해보시기 바랍니다.
지금까지 SageMaker에서 S3 버킷에 저장된 기계 학습 데이터를 사용해보는 작업을 알아보는 시간이었습니다....! 끝...!
'AWS > SageMaker' 카테고리의 다른 글
| [AWS] SageMaker에서 PyTorch를 활용한 기계 학습 환경 구축하기 (0) | 2023.07.04 |
|---|---|
| [AWS] SageMaker Notebook 알아보기 (0) | 2023.05.18 |
| [AWS] SageMaker Studio 알아보기 (0) | 2023.05.16 |
| [AWS] Amazon SageMaker 알아보기 (0) | 2023.04.27 |