Back

DISTCP

Syntax

hadoop distcp <source> <target>
hadoop distcp <source> <source> <source> <source> <target>

Confirm you can list the source and target namenodes.

hdfs dfs -ls hdfs://node1:8020
hdfs dfs -ls hdfs://node2:8020

Copy from remote(node1) to local

hadoop distcp hdfs://node1:8020/user/root/test_data distcp_target

Copy from local to remote(node2)

hadoop distcp test_data hdfs://node2:8020/user/root/distcp_target


Older HDP Versions

If you are copying from an older HDP cluster use hftp.
NOTE: Port and protocol are slightly different.

hadoop distcp hftp://node1:50070/user/root/test/sample.txt distcp_target