0

I'm trying to create and manage a TransferJob for Google Storage Transfer service via Google Deployment Manager. The aim is to copy objects from one Cloud Storage bucket to another daily.

Since I can't find any example code, just the API, I'm trying to figure out how to describe the desired resource.

---
resources:
  - type: storagetransfers.v1.transferJobs
    name: transferJobs/{{ properties['jobName'] }}
    properties:
      status: ENABLED
      projectId: {{ properties['projectId'] }}
      schedule:
        scheduleStartDate:
            day: 1
            month: 12
            year: 2020
        startTimeOfDay:
            hours: 1
            minutes: 0
      transferSpec:
        gcsDataSource:
          bucketName: {{ properties['sourceBucket'] }}
        gcsDataSink:
          bucketName: {{ properties['targetBucket'] }}
        transferOptions:
          overwriteObjectsAlreadyExistingInSink: TRUE

When I try to apply this resource via Deployment Manager I get an error:

ERROR: (gcloud.deployment-manager.deployments.update) Error in Operation [operation-1606821546786-5b5654b838085-34014f33-1566b3e6]: errors:

  • code: RESOURCE_NOT_FOUND message: The type [storagetransfer.v1.transferJobs] was not found.

I've tried many variations of the type but none work. I've spent hours scouring the documentation.

What is the correct type to use, and how could I have known?

Martijn Heemels
  • 7,438
  • 6
  • 39
  • 62
  • Have you primarily tried to create the Transfer job via the Cloud Console instead, as mentioned [here](https://cloud.google.com/storage-transfer/docs/managing-on-prem-jobs#create-transfer) or in the un-official documentation [here](https://cloud.netapp.com/blog/cloud-storage-transfer-service-for-google-cloud) – Artemis Georgakopoulou Dec 03 '20 at 15:53
  • @ArtemisGeorgakopoulou I'm not entirely sure what you mean. I'm able to create a Transfer Job via the Cloud Console just fine. Also via the REST API (i.e. curl requests to the GCP API). But I'm unable to create one via Deployment Manager and that's my goal. – Martijn Heemels Dec 04 '20 at 14:47
  • 1
    Hello, @Martijn Heemels, sorry for the confusion. I just wanted to make sure it is working as intended while using the Cloud Console. According to the [official documentation](https://cloud.google.com/deployment-manager/docs/configuration/supported-resource-types#apis_for_supported_resources) it seems that Storage Transfer Service is not supported with the Deployment Manager. – Artemis Georgakopoulou Dec 07 '20 at 14:14
  • @ArtemisGeorgakopoulou Thanks. Someone mentioned it's possible to create custom DM resource-types, based on the REST API. That would probably be enough for us, but I have no idea where to start. – Martijn Heemels Dec 10 '20 at 18:32
  • 2
    Hello, I am not able to find any documentation stating that you can create custom resource types in DM, what you can create is [composite types with templates](https://cloud.google.com/deployment-manager/docs/fundamentals#composite_types) based on the supported resource types. However, I did find a [Feature Request](https://issuetracker.google.com/153051980) asking for more resource types support, which you could follow and also add the resource type of Cloud Storage Transfer Service so that the engineering teams can evaluate your request. – Artemis Georgakopoulou Dec 11 '20 at 11:52

0 Answers0