Sneakernet

Sneakernet is an informal term for the transfer of electronic information by physically moving media such as magnetic tape, floppy disks, optical discs, USB flash drives or external hard drives between computers, rather than transmitting it over a computer network. The term, a tongue-in-cheek play on net(work) as in Internet or Ethernet, refers to walking in sneakers as the transport mechanism.[1]

A USB flash drive allows the transfer of data between individuals without use of the Internet.

Memory cards are a popular physical medium for transferring files and have become smaller in size as technology has advanced

Summary and background

Sneakernets, also known as trainnets or pigeonets, are in use throughout the computer world. Sneakernet may be used when computer networks are prohibitively expensive for the owner to maintain, in high-security environments where manual inspection (for re-classification of information) is necessary, where information needs to be shared between networks with different levels of security clearance, when data transfer is impractical due to bandwidth limitations, when a particular system is simply incompatible with the local network, unable to be connected, or when two systems are not on the same network at the same time. Because Sneakernets take advantage of physical media, security measures used for the transfer of sensitive information are respectively physical.

This form of data transfer is also used for peer-to-peer (or friend-to-friend) file sharing and has grown in popularity in metropolitan areas and college communities. The ease of this system has been facilitated by the availability of USB external hard drives, USB flash drives and portable music players.[2]

The United States Postal Service offers a Media Mail service for compact discs, among other items. This provides a viable mode of transport for long distance Sneakernet use. In fact, when mailing media with sufficiently high data density such as high capacity hard drives, the throughput (data transferred per unit of time) as well as the cost per unit of data transferred may compete favorably with networked methods of data transfer.[3]

Theory

From an information theory standpoint, sneakernets can achieve tremendous throughput, but they suffer from high latency (see Network performance). The throughput of the network is directly proportional to the size of the transmitted file(s). Latency is based on the amount of time it takes to fully process the request for information. Latency would include the time it takes to write the storage media and the time to travel from point A to point B.

For example: Alice requests Bob to send her a DVD (4.7 GB) worth of information. Over the Internet, the latency for the file request may be milliseconds—Alice starts receiving the information nearly immediately—but at a modest broadband download speed of 50 kB/s it may take up to a day to complete the transfer. On the other hand, Bob could burn a DVD and deliver it to Alice in an hour. The latency was an hour, but the throughput of the transfer is roughly equal to a transfer rate of 1305 kB/s.

If an Airbus A380 were filled with microSD cards each holding 512 gigabytes of storage capacity, the theoretical total storage space onboard would be approximately 91 exabytes. A 4h47m flight from New York to Los Angeles would work out to a data transport rate of well over 5 petabytes per second, although this does not account for the time required to write to and read from the cards, which would almost certainly be much longer than the duration of the flight.

Similarly, as of 2019 the highest capacity backup tape format available is LTO-8, with a capacity of 12 TB. If a tape of this capacity were sent by overnight mail and were to arrive around 20 hours after it was sent, the effective data rate would be 1328 Mbit/s. With networking technology, this magnitude of speed over this distance would be very difficult to attain without a costly dedicated connection as one would likely need to use several hops and have a connection that is not oversubscribed.

Sneakernets may also be used in tandem with computer network data transfer to increase data security. For example, a file or collection of files may be encrypted and sent over the Internet while the encryption key is printed and hand delivered or mailed. This method greatly reduces the possibility of an individual intercepting both the key and encrypted data.

Another way sneakernets are used together with network transfers is to provide an initial full backup as a base for incremental backups. In the case of a large (several terabyte) dataset that grows by just a few megabytes a day, the initial seeding of the data to be backed up would require an excessively long time upload over a network. One solution is to make a local copy of the data which is then physically relocated to the remote location and topped up with daily incremental backups over a network.

Usage examples

When Australia joined Usenet in 1983, it received articles via tapes sent from the United States to the University of Sydney, which disseminated data to dozens of other computers on the country's Unix network.[4]

The May 2011 raid of Osama bin Laden's compound in Abbottabad, Pakistan revealed that he used a series of USB thumb drives to store his email drafts. A courier of his would then take the saved emails to a nearby Internet cafe and send them out to the desired recipients.[5][6]

In September 2009, Durban company Unlimited IT reportedly pitted a messenger pigeon against South African ISP Telkom to transfer 4 GB of data 60 miles (97 km) from Howick to Durban. The pigeon, carrying the data on a memory stick, arrived in one hour eight minutes, with the data taking another hour to read from the memory stick. During the same two-hour period, only about 4.2% of the data had been transferred over the ADSL link.[7] A similar experiment was conducted in England in September 2010; the "pigeonnet" also proved superior.[8][9] In November 2009 the Australian comedy/current-affairs television program Hungry Beast repeated this experiment. The experiment had the team transfer a 700 MB file via three delivery methods to determine which was the fastest; A carrier pigeon with a microSD card, a car carrying a USB Stick, or a Telstra ADSL line. The data was to be transferred a distance of 132 km by road. The pigeon won the race with a time of approximately 1 hour 5 minutes, the car came in second at 2 hours 10 minutes, while the internet transfer did not finish, having dropped out a second time and not coming back.[10]

Google has used a sneakernet to transport large datasets, such as the 120 TB of data from the Hubble Space Telescope.[11][12] Users of Google Cloud can import their data into Google Cloud Storage through sneakernet.[13]

The SETI@home project uses a sneakernet to overcome bandwidth limitations: data recorded by the radio telescope in Arecibo, Puerto Rico is stored on magnetic tapes which are then shipped to Berkeley, California for processing. In 2005, Jim Gray reported sending hard drives and even "metal boxes with processors" to transport large amounts of data by postal mail.[14]

Wizzy Digital Courier provided Internet access to schools in South Africa with poor or no network connectivity by implementing UUCP on USB memory sticks. This allowed offline cached email transport and scoops of web pages to back-fill a web cache.[15]

Data analytics teams in the financial services sector often use sneakernets to transfer sensitive corporate information and information obtained from data mining, such as ledger entries, customer data and financial statistics. There are several reasons for this: firstly, sneakernets can generally provide very high security (and possibly more importantly, they are perceived to be secure) due to the impossibility of a Man-in-the-middle attack or packet sniffing; secondly, the volumes of data concerned are often extremely high; and thirdly, setting up secure network links between the client business and the analytics team's facilities is often either impossible or an extremely convoluted process.

Very Long Baseline Interferometry performed using the Very Long Baseline Array ships hard drives to a data reduction site in Socorro, New Mexico. They refer to their data transfer mechanism as "HDOA" (Hard Drives On Airplane).

The Rigsum Sherig Collection project[16] uses a sneakernet to distribute offline educational resources, including Kiwix and Khan Academy on a Stick,[17] to hundreds of schools and other educational institutional in the Kingdom of Bhutan. Many of the schools in Bhutan have computers or IT labs, but no Internet connection (or a very slow one).[18] The sneakernet, facilitated by teachers, distributes about 25 GB of free, open-source educational software to the schools, often using external hard disks.

North Korean dissidents have been known to smuggle flash drives filled with western movies and television shows, largely in an effort to inspire a cultural revolution.[19][20][21][22][23]

El Paquete Semanal is a roughly 1TB compilation of media, distributed weekly throughout Cuba via portable hard drives and USB memory sticks.[24]

In 2015 Amazon Web Services launched AWS Snowball, a 50 lb (23 kg), 50 TB device for transporting data to the AWS cloud;[25] and in 2016 AWS Snowmobile, a truck to transport up to 100 PB of data in one load.[26] For similar reasons, there is also a Google Transfer Appliance and an IBM Cloud Mass Data Migration device.[27]

Observation data from the Event Horizon Telescope is collected on hard drives which are transported by commercial freight airplanes[28] from the various telescopes to the MIT Haystack Observatory and the Max Planck Institute for Radio Astronomy, where the data are analyzed.[29]

In media

Non-fiction

There's a lot of band-width in a station wagon.

— Fred Gruenberger, Computing: A Second Course[30]

The very first problem in Andrew S. Tanenbaum's 1981 textbook Computer Networks asks the student to calculate the throughput of a St. Bernard carrying floppy disks.[31]

The first USENET citation is July 16, 1985 and it was widely considered an old joke already.

Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

— Andrew S. Tanenbaum[32]

Other alleged speakers included Tom Reidel, Warren Jackson, or Bob Sutterfield.

Although the station wagon transporting magnetic tapes is generally considered the canonical version, variants using trucks or Boeing 747s or C-5s and later storage technologies such as CD-ROMs, DVDs, Blu-rays, or SD Cards have frequently appeared.

Fiction

• The Terry Pratchett novel Going Postal (2004) includes a contest between a horse-drawn mail coach and the "Grand Trunk Clacks" (a semaphore line) to see which is faster to transmit the contents of a book to a remote destination.
• William Gibson's novel Spook Country (2007) also features sneakernets, with iPods being the storage device used to clandestinely move information.[33]

The "valuable data file" has become a common MacGuffin in action films and television programs and indeed the motif of the "valuable letter or documents" (pre-electronic information storage technology) dates back hundreds of years.

• The film Johnny Mnemonic (1995), based on the short story by William Gibson, stars Keanu Reeves as a digital courier with 320 GB of corporate data transported in his head.
• The film Live Free or Die Hard (2007) depicts a digital thief attempting to download 500 TB of financial data to a suitcase-sized package.
• In the episode "Amen" of The Newsroom (2012), associate producer Maggie Jordan, after being told "it's 2011, we don't run film manually", is told that the video is taking too long to render on the local machine. It cannot feasibly be transferred over the network in raw form and must be run to the newsroom manually on a USB drive.
• In the film Elysium (2013), produced, written, and directed by Neill Blomkamp, Matt Damon as Max Da Costa downloads sensitive data into his brain implant in a data heist.
• In the film Snowden (2016), the titular character is seen evading security by carrying a data chip concealed in a Rubik's Cube.
• In the James Bond film Skyfall (2016), MI6 sets up a sneakernet due to security concerns.
• A large portion of the game Fallout: New Vegas involves a courier recovering and delivering a stolen casino chip containing part of a program for an anti missile defense system.

Similar concepts

• Delay-tolerant networks, such as the Haggle project[34] at Cambridge University.
• IP over Avian Carriers (RFC 1149), an April Fools' Day RFC describing the transmission of messages via homing pigeon.

See also

• Air gap (networking)
• Darknet
• Data Mule
• Jargon File
• Meatspace
• Pod slurping
• Sideloading
• Twilight (warez)
• USB dead drop

References

1. "Oxford Dictionary". Retrieved 2016-09-09.
2. Boutin, Paul (2002-08-26). "Sneakernet Redux: Walk Your Data". Wired News. Archived from the original on November 19, 2009. Retrieved 8 June 2010.
3. Munroe, Randall. "FedEx Bandwidth". xkcd what if?. Retrieved 18 September 2019.
4. Marquis, Bret (1983-03-29). "Australia joins USENET". Newsgroup: net.news.newsite. [e-mail]. Retrieved 14 February 2016.
5. Apuzzo, Matt & Goldman, Adam (May 13, 2011). "How bin Laden emailed without being detected by US". The Washington Times. Associated Press. Retrieved June 29, 2012.CS1 maint: uses authors parameter (link)
6. McCullagh, Declan (May 13, 2011). "How bin Laden evaded the NSA: Sneakernet". Privacy Inc. CNET. Retrieved May 17, 2011.
7. "SA Pigeon 'Faster than broadband'". BBC News. September 10, 2009.
8. "BT feathers ruffled over pigeon-based file transfer caper". The Register. September 17, 2010.
9. Pigeon flies past broadband in data speed race, BBC News Technology, September 16, 2010
10. "The Great Australian Internet Challenge". ABC Television/Hungry Beast. 10 November 2009.
11. "Google helps terabyte data swaps". BBC News. March 7, 2007. Retrieved May 23, 2010.
12. Farivar, Cyrus (20 March 2007). "Google's Next-Gen of Sneakernet". Wired. Retrieved 5 February 2013.
13. "Offline Media Import / Export". Retrieved 29 January 2016.
14. "A Conversation with Jim Gray". ACM Queue. 1 (4). July 31, 2003. Archived from the original on December 5, 2009. Who would ever, in this time of the greatest interconnectivity in human history, go back to shipping bytes around via snail mail as a preferred means of data transfer?
15. "Seeking Riches From the Poor", Megan Lindow, 04.23.04, Wired.com
16. Rigsum Sherig Collection
17. Khan Academy on a Stick
18. "Only a Third of Government Schools Have Internet Access". Kuensel. April 18, 2013. Archived from the original on June 15, 2013.
19. Fighting The State, Without The Web: North Korea's Sneakernet Insurgency
20. https://www.wired.com/2015/03/north-korea/
21. How One Man Wants to Free North Korea With USB Drives and Pirated Movies
22. North Korea’s Secret Movie Bootleggers: How Western Films Make It Into the Hermit Kingdom
23. Balloon activist sends 'thousands of copies' of The Interview to North Korea
24. In Cuba, An Underground Network Armed With USB Drives Does The Work Of Google And YouTube
25. Kastrenakes, Jacob (October 7, 2015). "Amazon made a huge plastic box called Snowball so people can ship data to the cloud". The Verge. Retrieved October 8, 2015.
26. Dignan, Larry (November 30, 2016). "AWS' Snowmobile data transport truck highlights why cloud giant is so damn disruptive".
27. Sharwood, Simon (19 September 2017). "IBM packs 120TB into a carry-on bag, for snow-balling cloud uploads". The Register. Retrieved 19 September 2017.
28. "The Hidden Shipping and Handling Behind That Black-Hole Picture". The Atlantic. Retrieved 2019-04-14.
29. Mearian, Lucas (18 August 2015). "Massive telescope array aims for black hole, gets gusher of data". Computerworld. Retrieved 2015-08-21.
30. Gruenberger, Fred (1971). Computing: A Second Course. San Francisco: Canfield Press. p. 138. ISBN 978-0063834057. Retrieved 24 January 2017.
31. "Updated Textbook Explores Theoretical Basis of Networks". InfoWorld. 6 Feb 1989. Retrieved 16 April 2019.
32. Tanenbaum, Andrew S. (1989). Computer Networks. New Jersey: Prentice-Hall. p. 57. ISBN 0-13-166836-6.
33. Poole, Steven (August 18, 2007). "Sign language". The Guardian. London: Guardian Media Group. Retrieved September 24, 2009.
34. Ben Hui (March 1, 2006). "Haggle". University of Cambridge Computer Laboratory. Archived from the original on June 28, 2015.