Sneakernet


Sneakernet is an informal term for the transfer of electronic information by physically moving media such as magnetic tape, floppy disks, optical discs, USB flash drives or external hard drives between computers, rather than transmitting it over a computer network. The term, a tongue-in-cheek play on net as in Internet or Ethernet, refers to walking in sneakers as the transport mechanism.

Summary and background

Sneakernets, also known as trainnets or pigeonets, are in use throughout the computer world. Sneakernet may be used when computer networks are prohibitively expensive for the owner to maintain, in high-security environments where manual inspection is necessary, where information needs to be shared between networks with different levels of security clearance, when data transfer is impractical due to bandwidth limitations, when a particular system is simply incompatible with the local network, unable to be connected, or when two systems are not on the same network at the same time. Because Sneakernets take advantage of physical media, security measures used for the transfer of sensitive information are respectively physical.
This form of data transfer is also used for peer-to-peer file sharing and has grown in popularity in metropolitan areas and college communities. The ease of this system has been facilitated by the availability of USB external hard drives, USB flash drives and portable music players.
The United States Postal Service offers a Media Mail service for compact discs, among other items. This provides a viable mode of transport for long distance Sneakernet use. In fact, when mailing media with sufficiently high data density such as high capacity hard drives, the throughput as well as the cost per unit of data transferred may compete favorably with networked methods of data transfer.

Theory

From an information theory standpoint, sneakernets can achieve tremendous throughput, but they suffer from high latency. The throughput of the network is directly proportional to the size of the transmitted file. Latency is based on the amount of time it takes to fully process the request for information. Latency would include the time it takes to write the storage media and the time to travel from point A to point B.
For example: Alice requests Bob to send her a DVD worth of information. Over the Internet, the latency for the file request may be milliseconds—Alice starts receiving the information nearly immediately—but at a modest broadband download speed of 50 kB/s it may take up to a day to complete the transfer. On the other hand, Bob could burn a DVD and deliver it to Alice in an hour. The latency was an hour, but the throughput of the transfer is roughly equal to a transfer rate of 1305 kB/s.
If an Airbus A380 were filled with microSD cards each holding 512 gigabytes of storage capacity, the theoretical total storage space onboard would be approximately 91 exabytes. A 4h47m flight from New York to Los Angeles would work out to a data transport rate of well over 5 petabytes per second, although this does not account for the time required to write to and read from the cards, which would almost certainly be much longer than the duration of the flight.
Similarly, the highest capacity backup tape format available is LTO-8, with a capacity of 12 TB. If a tape of this capacity were sent by overnight mail and were to arrive around 20 hours after it was sent, the effective data rate would be 1328 Mbit/s. With networking technology, this magnitude of speed over this distance would be very difficult to attain without a costly dedicated connection as one would likely need to use several hops and have a connection that is not oversubscribed.
Sneakernets may also be used in tandem with computer network data transfer to increase data security. For example, a file or collection of files may be encrypted and sent over the Internet while the encryption key is printed and hand delivered or mailed. This method greatly reduces the possibility of an individual intercepting both the key and encrypted data.
Another way sneakernets are used together with network transfers is to provide an initial full backup as a base for incremental backups. In the case of a large dataset that grows by just a few megabytes a day, the initial seeding of the data to be backed up would require an excessively long time upload over a network. One solution is to make a local copy of the data which is then physically relocated to the remote location and topped up with daily incremental backups over a network.

Usage examples

When Australia joined Usenet in 1983, it received articles via tapes sent from the United States to the University of Sydney, which disseminated data to dozens of other computers on the country's Unix network.
The May 2011 raid of Osama bin Laden's compound in Abbottabad, Pakistan revealed that he used a series of USB thumb drives to store his email drafts. A courier of his would then take the saved emails to a nearby Internet cafe and send them out to the desired recipients.
In September 2009, Durban company Unlimited IT reportedly pitted a messenger pigeon against South African ISP Telkom to transfer 4 GB of data from Howick to Durban. The pigeon, carrying the data on a memory stick, arrived in one hour eight minutes, with the data taking another hour to read from the memory stick. During the same two-hour period, only about 4.2% of the data had been transferred over the ADSL link. A similar experiment was conducted in England in September 2010; the "pigeonnet" also proved superior. In November 2009 the Australian comedy/current-affairs television program Hungry Beast repeated this experiment. The experiment had the team transfer a 700 MB file via three delivery methods to determine which was the fastest; A carrier pigeon with a microSD card, a car carrying a USB Stick, or a Telstra ADSL line. The data was to be transferred a distance of 132 km by road. The pigeon won the race with a time of approximately 1 hour 5 minutes, the car came in second at 2 hours 10 minutes, while the internet transfer did not finish, having dropped out a second time and not coming back.
Google has used a sneakernet to transport large datasets, such as the 120 TB of data from the Hubble Space Telescope. Users of Google Cloud can import their data into Google Cloud Storage through sneakernet.
The SETI@home project uses a sneakernet to overcome bandwidth limitations: data recorded by the radio telescope in Arecibo, Puerto Rico is stored on magnetic tapes which are then shipped to Berkeley, California for processing. In 2005, Jim Gray reported sending hard drives and even "metal boxes with processors" to transport large amounts of data by postal mail.
Wizzy Digital Courier provided Internet access to schools in South Africa with poor or no network connectivity by implementing UUCP on USB memory sticks. This allowed offline cached email transport and scoops of web pages to back-fill a web cache.
Data analytics teams in the financial services sector often use sneakernets to transfer sensitive corporate information and information obtained from data mining, such as ledger entries, customer data and financial statistics. There are several reasons for this: firstly, sneakernets can generally provide very high security due to the impossibility of a Man-in-the-middle attack or packet sniffing; secondly, the volumes of data concerned are often extremely high; and thirdly, setting up secure network links between the client business and the analytics team's facilities is often either impossible or an extremely convoluted process.
Very Long Baseline Interferometry performed using the Very Long Baseline Array ships hard drives to a data reduction site in Socorro, New Mexico. They refer to their data transfer mechanism as "HDOA".
The Rigsum Sherig Collection project uses a sneakernet to distribute offline educational resources, including Kiwix and Khan Academy on a Stick, to hundreds of schools and other educational institutional in the Kingdom of Bhutan. Many of the schools in Bhutan have computers or IT labs, but no Internet connection. The sneakernet, facilitated by teachers, distributes about 25 GB of free, open-source educational software to the schools, often using external hard disks.
North Korean dissidents have been known to smuggle flash drives filled with western movies and television shows, largely in an effort to inspire a cultural revolution.
El Paquete Semanal is a roughly 1TB compilation of media, distributed weekly throughout Cuba via portable hard drives and USB memory sticks.
In 2015 Amazon Web Services launched AWS Snowball, a, 50 TB device for transporting data to the AWS cloud; and in 2016 AWS Snowmobile, a truck to transport up to 100 PB of data in one load. For similar reasons, there is also a Google Transfer Appliance and an IBM Cloud Mass Data Migration device.
Observation data from the Event Horizon Telescope is collected on hard drives which are transported by commercial freight airplanes from the various telescopes to the MIT Haystack Observatory and the Max Planck Institute for Radio Astronomy, where the data are analyzed.

In media

Non-fiction

The very first problem in Andrew S. Tanenbaum's 1981 textbook Computer Networks asks the student to calculate the throughput of a St. Bernard carrying floppy disks.
The first USENET citation is July 16, 1985 and it was widely considered an old joke already.
Other alleged speakers included Tom Reidel, Warren Jackson, or Bob Sutterfield.
Although the station wagon transporting magnetic tapes is generally considered the canonical version, variants using trucks or Boeing 747s or C-5s and later storage technologies such as CD-ROMs, DVDs, Blu-rays, or SD Cards have frequently appeared.

Fiction

The "valuable data file" has become a common MacGuffin in action films and television programs and indeed the motif of the "valuable letter or documents" dates back hundreds of years.