How To Transfer Files To and From Headless Linux Systems

2023-11-29 | By Maker.io Staff

Sometimes, you may want to send files such as a custom program or script to a headless Linux computer. In other cases, you may want to retrieve data from a headless computer, as is often the case with log files created by the headless system. However, the lack of a keyboard, mouse, and monitor might result in wondering how to copy files to and from a headless system. Therefore, this article discusses two standard methods of transferring files between a headless Linux computer and other computers.

Download Files to a Headless System

When using a computer with a monitor, keyboard, and mouse, you can open a web browser, enter a resource’s URL, and download files using the browser. A straightforward approach would be to host a VNC server and share the graphical user interface of the headless computer. Then, you could use a client computer with a monitor to download files. However, in many situations, the headless computer doesn’t run a graphical OS interface to preserve system resources, so using a standard web browser is out of reach.

You can establish an SSH connection to the headless computer and utilize a built-in tool called wget to download files, such as images and executable programs available on the Internet. In addition, wget allows automated downloads — for example, when used in bash scripts, which can be helpful when automating tasks on a computer, such as updating an installed program once every month.

Either way, wget’s basic syntax is straightforward and easy to remember:

Copy Code

wget [options] URL

When not supplied with any options, wget attempts to download the resource it finds at the given URL if the resource exists. The following command, as an example, downloads the header image from DigiKey's website and stores it under the same file name that the image has on the web server:

Copy Code

wget https://www.digikey.com/-/media/Images/Header/logo_dk.png

The optional -P flag instructs wget to store the downloaded file in a different path:

Copy Code

wget -P path/to/target/directory https://example.com/file.zip

Wget displays a status bar to inform you about the progress and how long the download will approximately take. In addition, wget can resume downloads that were interrupted when supplied with the -c flag:

How To Transfer Files To and From Headless Linux Systems In this simulated example, I aborted the download twice and then used the -c flag to tell wget to resume the download without creating a new file.

Note that the output may not always be required, such as when running wget as part of an automation script. In that case, the -q option suppresses all status messages. Finally, wget supports HTTP-based authentication, and you can supply the tool with your username and password using the following options:

Copy Code

wget --user=username --password=pass https://page.com/prv/img.png

Note that these are only some of the most vital options for wget, and the tool’s man page offers more information about the program and supported options.

Moving Files Between a Headless System and a Local Computer

While the first method is useful for downloading files hosted on the Internet or a private server, it doesn’t allow retrieving files from the headless machine or sending local files directly to the Linux system without hosting them first. Another program called scp, however, allows the exchange of files and entire directories between two computers in a network. The tool’s basic syntax is as follows:

Copy Code

scp [options] source destination

Under the hood, scp uses the ssh protocol to authenticate users and transmit files securely. The user running the command must prove they have permission to access the target and source destinations, for which they can provide scp with the remote computer's username. When accessing a restricted resource, scp asks for the user's password.

There are two use cases you should know. First, you can use the following command to transfer a local file to a remote computer:

Copy Code

scp local_file remote_user@remote_computer:remote_path

Conversely, the following command retrieves a remote file from another computer and copies it to the local machine:

Copy Code

scp remote_user@remote_computer:remote_path local_file

To make it more concrete, suppose you wanted to copy a file called my-file.txt from the machine you’re currently using to a Raspberry Pi located within your network. You want to perform this action as the standard pi user and place the copy in the pi user’s home directory. You can do so using the following command:

Copy Code

scp my-file.txt pi@raspberrypi.local:~

Similarly, if you wanted to retrieve a log file from the Raspberry Pi and store it on your local machine, you could write:

Copy Code

scp pi@raspberrypi.local:~/log.txt ~/local_folder/pi_log_1.txt

Note that scp is available on all Unix-like systems. However, Windows users can use Windows Subsystem for Linux, a Unix-like bash (e.g., the git bash), or dedicated programs, such as WinSCP.

Summary

Wget and scp are two vital commands you should have in your repertoire. Wget lets you download resources hosted on servers to a headless Unix-like computer without a graphical UI or additional programs. The tool supports many features you might expect to find in a file downloader, and it can even resume previously aborted or paused downloads.

In contrast, you can use scp to securely and conveniently transfer files between computers in a network. The basic syntax is always the same, and it consists of the scp command itself, followed by the source file, and then by the destination. Note that both tools support many additional options not covered in this article, and you can always refer to the program’s respective man pages for more details.