Issue
What's the best way to copy big query table from one project to another using Dataflow. I did that using CopyJobConfiguration but, for that we always need to raise job creation access on the other project. So I am looking for the best, optimal and economical solution to copy data. On GCP I couldn't find the recommended approach other than JobCopyConfiguration.
Solution
I think the best way is to directly using Bigquery to copy a table from one project to another, no need to Dataflow in this case if you don't have business rules and tranformations.
You can use for example gcloud sdk and bq :
bq cp project1:dataset1.table1 project2:dataset2.table2
You can also directly use the Bigquery console from the GUI to do that.
You must using an identity that has the needed privileges on the two projects.
Answered By - Mazlum Tosun
Answer Checked By - Clifford M. (JavaFixing Volunteer)
