Categories
Design

How to use pandoc to convert files on the linux command line

Last updated June 26, 2021 By Bill Dyer 4 Comments

In an earlier article, I covered the procedure to batch convert a handful of Markdown files to HTML using pandoc. In that article, multiple HTML files were created, but pandoc can do much more. It has been called “the Swiss army knife” of document conversion – and with good reason. There isn’t a lot that it can’t do.

Pandoc can covert .docx, .odt, .html, .epub, LaTeX, DocBook, etc. to these and other formats, such as JATS, TEI Simple, AsciiDoc, and more.

Yes, this means that pandoc can convert .docx files to .pdf and .html, but you may be thinking: “Word can export files to .pdf and .html too. Why would I need pandoc?”

You would have a good point there, but since pandoc can convert so many formats, it could well become your go-to tool for all of your conversion tasks. For example, many of us know that Markdown editors can export its Markdown files to .html. With pandoc, Markdown files can be converted to numerous other formats as well.

I rarely have Markdown export to HTML; I normally let pandoc do it.

Converting File Formats with Pandoc

How to use pandoc to convert files on the linux command line

Here, I will convert Markdown files into a few different formats. I do almost all of my writing using Markdown syntax, but I often have to convert to another format: .docx files are usually required for school work, .html for web pages that I create – and for .epub work, .pdf for flyers and handouts, and even an occasional TEI Simple file for a university digital humanities project. Pandoc can handle all of these, and more, easily.

First, you need to install pandoc. Also, to create .pdf files, LaTeX will be needed as well. The package I prefer is TeX Live.

Note: If you would like to try out pandoc before installing it, there is an online try-out page at: http://pandoc.org/try/

Installing pandoc and texlive

Users of Ubuntu and other Debian distros can type the following commands in the terminal:

Notice on the second line, you are installing pandoc and texlive in one shot. apt-get command will have no problem with this, but go get some coffee; this may take a few minutes.

Getting to Conversion

Once pandoc and texlive are installed, you can burn through some work!

The sample document for this project will be an article that was first published in the North American Review in December of 1894, and is titled: “How To Repel Train Robbers”. The Markdown file that I will be using was created some time ago as part of a restoration project.

The file: how_to_repel_train_robbers.md is located in my Documents directory, in a sub-directory named samples. Here is what it looks like in Ghostwriter.

How to use pandoc to convert files on the linux command lineMarkdown file in Ghostwriter

I want to create .docx, .pdf, and .html versions of this file.

The First Conversion

I’ll start with making a .pdf copy first, since I went through the trouble of installing a LaTeX package.

/Documents/samples/ directory, I type the following to create a .pdf file:

The above command will create a file called htrtr.pdf from the how_to_repel_train_robbers.md file. The reason I used htrtr as a name was that it is shorter than how_to_repel_train_robbers – htrtr is the first letter of each word in the long title.

Here is a snapshot of the .pdf file once it is made:

The Second Conversion

Next, I want to create a .docx file. The command is almost identical to the one I used to create the .pdf and it is:

In no time, a .docx file is created. Here is what it looks like in Libre Writer:

The Third Conversion

I may want to post this on the web, so a web page would be nice. I will create a .html file with this command:

Again, the command to create it is very much like the last two conversions. Here is what the .html file looks like in a browser:

How to use pandoc to convert files on the linux command lineConverted HTML file viewed in Firefox

Noticed Anything Yet?

Let’s look at the past commands again. They were:

The only thing different about these three commands is the extension next to htrtr. This gives you a hint that pandoc relies on the extension of the output filename you provide.

Conclusion

Pandoc can do far more than the three little conversions done here. If you write with a preferred format, but need to convert the file to another format, chances are great that pandoc will be able to do it for you.

What would you do with this? Would you automate this? What if you had a web site that had articles for your readers to download? You could modify these little commands to work as a script and your readers could decide which format they would like. You could offer .docx, .pdf, .odt, .epub, or more. Your readers choose, the proper conversion script runs, and your readers download their file. It can be done.

Like what you read? Please share it with others.

Last updated April 3, 2021 By Bill Dyer 5 Comments

Many times, when I use Markdown, I work on one file and when I’m done with it, I convert it to HTML or some other format. Occasionally, I have to create a few files. When I do work with more than one Markdown file, I usually wait until I have finished them before I convert them.

I use pandoc to convert files, and it’s possible convert all the Markdown files in one shot.

Markdown can convert its files to .html, but if there’s a chance that I will have to convert to other formats like epub, pandoc is the tool to use. I prefer to use the command line, so I will cover that first, but you can also do this in VSCodium without the command line. I’ll cover that too.

Converting multiple Markdown files to another format with Pandoc [command line method]

To get started quickly, Ubuntu, and other Debian distros can type the following commands in the terminal:

In this example, I have four Markdown files in a directory called md_test.

There are no HTML files yet. Now I’ll use Pandoc to do its magic on the collection of files. To do this, I run a one-line command that:

  • calls pandoc
  • reads the .md files and exports them as .html

This is the command:

If you are not aware already, ; is used for running multiple commands at once in Linux.

Here’s what the display looks like once I have executed the command:

Let me use the ls command once more to see if HTML files were created:

The conversion was a success, and you have four HTML files ready to go on the Web server.

Pandoc is quite versatile and you can convert the markdown files to some other supported format by specifying the extension of the output files. You can understand why it is considered among the best open source tools for writers.

Recommended Read:

How to use pandoc to convert files on the linux command line

11 Best Markdown Editors for Linux

A list of best Markdown Editors for Linux distributions that not only look good but are also feature rich.

Converting Markdown files to HTML using VSCodium [GUI method]

Like I’ve said earlier, I normally use the command line, but I don’t always use it for batch conversions, and you don’t have to either. VSCode or VSCodium can do the job. You just need to add one extension, called: Markdown-All-in-One which will allow you to convert more than one Markdown file in one run.

There are two ways to install the extension:

  • VSCodium’s terminal
  • VSCodium’s plug-in manager

To install the extension through VSCodium’s terminal:

  1. Click on Terminal on the menu bar. The terminal panel will open
  2. Type, or copy-and-paste, the following command in the terminal:

Note: If you’re using VSCode instead of VSCodium, replace the word, codium , in the above command, with code

How to use pandoc to convert files on the linux command line

The second way to install is through VSCodium’s plug-in, or extension, manager:

  1. Click on the blocks on the left side of the VSCodium window. A list of extensions will appear. At the top of the list, there will be a search bar.
  2. In the search bar, type: Markdown All in One . The extension will be listed at the top of the list. Click on the Install button to install it. If it is already installed, a gear icon will appear in place of the install button.

How to use pandoc to convert files on the linux command line

Once the extension is installed, you can open the folder that contains the Markdown files you want to convert.

Click on the paper icon located on the left side of the VSCodium window. You’ll be given the opportunity to choose your folder. Once a folder is open, you’ll need to open at least one file. You can open as many files as you want, but one is the minimum.

Once a file is open, bring up the Command Palette by pressing CTRL+SHIFT+P . Then, start typing Markdown in the search bar that will appear. As you do this, a list of Markdown related commands will appear. One of these will be Markdown All in One: Print documents to HTML command. Click on that one.

How to use pandoc to convert files on the linux command line

You’ll be asked to choose a folder containing the files. This is so an output directory (called out ) can be made and this is where the HTML files will go. The image below shows that the HTML was made after exporting the Markdown documents. From here, you can open, view, and edit the HTML as you wish.

How to use pandoc to convert files on the linux command line

By waiting to convert your Markdown files, you can concentrate more on writing. Conversion to HTML can come when you’re ready – and you have two ways to get that done.

Like what you read? Please share it with others.

I would like to install a command line tool within a Docker image in order to quickly convert *html files into *pdf files.

I am surprised there is not a Unix tool to do something like this.

How to use pandoc to convert files on the linux command line

6 Answers 6

pandoc is a great command-line tool for file format conversion.

The disadvantage is for PDF output, you’ll need LaTeX. The usage is

If you don’t have LaTeX installed, then I recommend htmldoc.

By default, pandoc will use LaTeX to create the PDF, which requires that a LaTeX engine be installed.

Alternatively, pandoc can use ConTeXt, pdfroff, or any of the following HTML/CSS-to-PDF-engines, to create a PDF: wkhtmltopdf, weasyprint or prince. To do this, specify an output file with a .pdf extension, as before, but add the –pdf-engine option or -t context, -t html, or -t ms to the command line (-t html defaults to –pdf-engine=wkhtmltopdf).

How to use pandoc to convert files on the linux command line

You can also try wkhtmltopdf, usage and installation is pretty straightforward.

weasyprint is an option. A possible drawback is that you’ll need python on your machine.

Install:

Convert:

I’ve been successfully using the 1.8 branch of HTMLDOC for years. I put it in a commercial system that has since generated hundreds of thousands of reports since 2003.

It’s not super-versatile, but it is very efficient and reliable. It’s limited to a basic set of postscript fonts.

It does not support CSS, but instead uses a special HTML comment directive set to control PDF specific aspects.

The source code is not too difficult to read and edit if you need to add custom facilities, if you’re comfortable with C. It is compiled with GCC or Visual Studio, depending on your target platform.

Note that the HTML does not need to be in a file. You can generate it dynamically from a URL, php or aspx etc. You can also hook it up in your web server for generate a PDF file dynamically.

In my use case it generates a PDF file from an asp page which then gets attached to an email, instead of sending the HTML to the printer and the letter stuffing machine; it’s a kind of print spooler.

Command-line Markdown tools are fast, powerful, and flexible. Here are four utilities to try.

How to use pandoc to convert files on the linux command line

Subscribe now

Get the highlights in your inbox every week.

When it comes to working with files formatted with Markdown, command-line tools rule the roost. They’re light, fast, powerful, and flexible, and most of them follow the Unix philosophy of doing one thing well.

Take a look at four utilities that can help you work more efficiently with Markdown files at the command line.

mdless

If you’ve hung around the Linux command line for a while, you’re probably familiar with a text-file viewer called less. Sure, you can use less to view Markdown files—but the result is a tad dull. How can you view Markdown files with a bit of pizzazz in a terminal window? By using mdless.

mdless.png

How to use pandoc to convert files on the linux command line

You can move around using the arrow keys on your keyboard, and mdless packs quite a good search function.

Not only does mdless display text, but it also renders formatting like headings, bold, and italics. It can also display tables and do syntax highlighting of code blocks. You can also customize the look and feel of mdless by creating one or more theme files.

Markdown lint tool

When you’re typing quickly, you make mistakes. If you miss a bit of formatting when using Markdown (or any other markup language), it can cause problems when you convert your file to another format.

Coders often use tools called linters to check for correct syntax. You can do the same for Markdown using the Markdown lint tool.

When you run this tool over a file that’s formatted with Markdown, it checks the formatting against a set of rules. Those rules govern the structure of a document, including the order of header levels, incorrect indentation and spacing, problems with code blocks, the existence of HTML in a file, and more.

mdl.png

How to use pandoc to convert files on the linux command line

The rules can be a bit strict. But running the Markdown lint tool over a file before converting it to another format can prevent the grief that comes from bad or inconsistent formatting.

mdmerge

  • Linux commands cheat sheet
  • Advanced Linux commands cheat sheet
  • Free online course: RHEL Technical Overview
  • Linux networking cheat sheet
  • SELinux cheat sheet
  • Linux common commands cheat sheet
  • What are Linux containers?
  • Our latest Linux articles

I wish I’d known about mdmerge before I started that project. It would have saved me a lot of time and energy.

mdmerge, as you’ve probably guessed from its name, combines two or more Markdown files into a single file. You don’t need to type the names of the files at the command line. Instead, you can add them to a file called book.txt and use that as an input file for mdmerge.

That’s not all mdmerge can do. You can add a reference to another document—either one formatted with Markdown or a piece of source code—and pull it into your main document. That enables you to create master documents that you can tailor to specific audiences.

mdmerge isn’t one of those utilities you’ll use all the time. When you need it, you’ll be glad it’s on your hard drive.

bashblog

bashblog isn’t strictly a tool for working with Markdown. It takes files that are formatted using Markdown and uses them to build a simple blog or website. Think of bashblog as a static site generator, but one that doesn’t have a bunch of fragile dependencies. Just about everything you need is in a shell script weighing just under 50KB.

To use bashblog, all you need is a Markdown processor installed on your computer. From there, you edit the shell script to add information about your blog—for example, its title, your name, your social media links, and the like. Then run the script. A new post opens in your default text editor. Start typing.

After you save a post, you can publish it or save it as a draft. If you choose to publish the post, bashblog generates your blog, posts and all, as a set of HTML files that you can upload to a web server.

Out of the box, your blog is bland but serviceable. You can edit the site’s CSS file to give it a look and feel all your own.

bashblog.png

How to use pandoc to convert files on the linux command line

What about Pandoc?

Sure, Pandoc is a very powerful tool for converting files formatted with Markdown to other markup languages. But there’s more to working with Markdown at the command line than Pandoc.

    Home
  1. News
  2. Software
  3. Pandoc 2.14.1

This website can use cookies to improve the user experience

This website can use cookies to improve the user experience and to provide certain services and functions to users. Cookies contain small amounts of information (such as login information and user preferences) and will be stored on your device.

Pandoc 2.14.1

Software 119950 2021-07-19 12:00 by Timothy Tibbetts

Pandoc is a command-line utility that can convert files from one markup format into another.

Pandoc’s enhanced version of Markdown includes syntax for footnotes, tables, flexible ordered lists, definition lists, fenced code blocks, superscripts and subscripts, strikeout, metadata blocks, automatic tables of contents, embedded LaTeX math, citations, and Markdown inside HTML block elements. (These enhancements, described further under Pandoc’s Markdown, can be disabled using the markdown_strict input or output format.)

In contrast to most existing tools for converting Markdown to HTML, which use regex substitutions, Pandoc has a modular design: it consists of a set of readers, which parse text in a given format and produce a native representation of the document, and a set of writers, which convert this native representation into a target format. Thus, adding an input or output format requires only adding a reader or writer.

Because Pandoc’s intermediate representation of a document is less expressive than many of the formats it converts between, one should not expect perfect conversions between every format and every other. Pandoc attempts to preserve the structural elements of a document but not formatting details such as margin size. And some document elements, such as complex tables, may not fit into Pandoc’s simple document model. While conversions from Pandoc’s Markdown to all formats aspire to be perfect, conversions from formats more expressive than Pandoc’s Markdown can be expected to be lossy.

Similar:
The Ultimate List of Every Known Command Prompt and PowerShell Commands
How To Run PowerShell or the Command Prompt as Administrator
How to Reset PowerShell and Command Prompt
How to Add ‘Open PowerShell Window Here’ as Administrator Context Menu

PDF or the Portable Document Format is mostly our first choice when it comes to printing, sharing and emailing documents, especially the larger ones. For Windows and macOS, you might be very much familiar, and also dependent on, the widely used Acrobat products for pdf file creation, viewing, and editing. Unfortunately, there is no default pdf creator available on your Linux systems. You can, however, use the LibreOffice shell tools to create PDF files in Debian. In this article, we will explain how you can use the Debian command line, the Terminal, in order to convert and batch convert .doc and .docx files to their pdf versions.

Why the Command Line?

If you are a Terminal-savvy person, you wouldn’t want to leave the comfort of the command line and go somewhere else to do any of your daily technical activities. There is always a way to do almost all of our stuff right inside the Terminal. So, why should pdf conversion be any different! Using the Terminal makes certain tasks more efficient, and even faster. The command-line tools do not use too many resources and thus form great alternatives to the widely used graphical applications, especially if you are stuck up with older hardware.

We have run the commands and procedures mentioned in this article on a Debian 10 Buster system.

Using the LibreOffice CLI Lowriter for pdf conversion

LibreOffice Write is part of the LibreOffice package and is mostly available by default in most Linux distros. If your system lacks it, you can easily install it from the Debian Software Manager:

How to use pandoc to convert files on the linux command line

Here, we will be making use of the CLI of the same in order to convert our documents to pdfs.

Here is how you can use the LOwriter from the command line:

Open your Debian command line, the Terminal, through the Debian Application Launcher search as follows:

How to use pandoc to convert files on the linux command line

The Application Launcher can be accessed using the Super/Windows key.

Please make sure that lowriter is installed on your system by running the following command:

How to use pandoc to convert files on the linux command line

Convert a single ODT, DOC or DOCX file to PDF

Use the following syntax in order to convert a single file located in your current directory:

Here is how I converted a .docx file to pdf located in my Downloads folder.

In Case of Error:

If you get the following error while trying to convert the file:

Then, try installing the libreoffice-java-common package as follows:

As you can see below, when I listed the contents of my current folder, after the pdf conversion process, through the ls command, I could see the newly converted pdf file as well.

Batch Conversion of DOC and DOCX or ODT files to pdf

Use the following syntax to batch convert all .doc or .docx files to pdf, located in your current directory:

This is how you can make use of the LibreOffice CLI to convert your documents from .doc and .docx to pdfs. No extra installations or lengthy procedures are required and you have exactly what you need; a .doc/.docx to pdf conversion right through the Debian command line.

  • ← How to Install and Configure LAMP Server on Ubuntu
  • How to change the Time Limit for a Sudo Session →

Karim Buzdar

About the Author: Karim Buzdar holds a degree in telecommunication engineering and holds several sysadmin certifications. As an IT engineer and technical author, he writes for various web sites. You can reach Karim on LinkedIn

In the last article, we learned how Markdown can quickly help you produce clean HTML code to be used in a website or blog. But what if you also want to produce an ebook using the same content as you have on the web? While the Markdown tool set is targeted at creating web content, there is another tool that allows you to take Markdown and turn it into OpenOffice/LibreOffice documents, PDF’s, or even e-books suitable for a Kindle or other e-reader – Pandoc.

Installing the pandoc package on an Ubuntu system is dead simple with the following command:

Once installed, you can immediately use Pandoc in place of Markdown to create HTML with the following command:

The syntax and flags are as follow:

  • “-r” – read format
  • “-w” – write format
  • “-o” – filename of the output

What the above command do is to read from a markdown file and output the file in HTML format with the same filename.

The above example outputs the file in HTML format, but you can use Pandoc to generate other formats as well.

Open Document Text Format (ODT)

If you’ll need to exchange your document with people using a more generic office suite, such as OpenOffice/LibreOffice or Microsoft Office, you can convert it to ODT format using Pandoc. If you think you’ll do this often, it’s useful to set up a template beforehand. Firstly, create a simple document (such as a header and a line or two of text) in Markdown and convert it to ODT with the following command:

Then, open the “pandoctemplate.odt” file in Open/LibreOffice to change the fonts, spacing, margins, etc… to your liking. Be sure to use Styles to configure this – some details on the use of styles are available here. Once your document is set up to your liking, you can use it as a template for creating ODT files from Markdown in the future by adding it to the above command:

Now when you convert a Markdown file to ODT, it will automatically be formatted with the styles you have created earlier. Pandoc also supports conversion to the new (version 2007 and later) Microsoft Word format with the flag “ –reference-docx=templatefile.docx “.

How to use pandoc to convert files on the linux command line

How to use pandoc to convert files on the linux command line

How to use pandoc to convert files on the linux command line

Portable Document Format (PDF)

When I need to generate PDF files from Markdown, I’ll most often convert it ODT, and use either LibreOffice’s Export to PDF function, or if it’s a large group of files, the “unoconv” command line utility. If you’re a LaTeX user, and have a number of packages installed (this section of the Pandoc documentation describes what’s required), you can output PDF’s with the following command:

Note the absence of the “-w” flag in this case.

ePub e-Books

To publish e-books suitable for most electronic readers (ePub is a format handled by almost all readers), you may want to have some items specific to that format prepared in advance. These include:

  • A stylesheet, written in CSS, that describes how the ePub will look
  • Metadata, such as the creator, description, rights to the work, and language
  • A cover image

If you don’t have these, however, Pandoc will use some reasonable defaults. The following command will convert your Markdown document to an ePub:

Additional Markdown Tips

Here are some additional tips and tricks I use in the course of using Markdown for my writing tasks:

  • Since it’s plain text, if you use DropBox to keep files in sync between devices, you can use the built-in text editor to create or update your Markdown documents on the Web. There are also editors available for Linux (I happen to like ReText a lot) and Android (I’ve been switching between Writer and the code editor DroidEdit lately).
  • Also, since it’s plain text, concurrent versioning systems (such as Subversion) do an excellent job of tracking versions and showing the differences between them.
  • Once you’ve converted a couple of documents, and know which flags you need for all the formats you want, you can create a simple shell script that will output them all at once.

I’ve found Markdown to be an excellent way to draft content, in a “distraction-free” environment (most plain text editors are), that supports output to multiple formats, yet doesn’t require any dedicated applications.

Aaron is an interactive business analyst, information architect, and project manager who has been using Linux since the days of Caldera. A KDE and Android fanboy, he’ll sit down and install anything at any time, just to see if he can make it work. He has a special interest in integration of Linux desktops with other systems, such as Android, small business applications and webapps, and even paper.

  • Facebook
  • Tweet

One comment

I used Markdown (using OSX, Sublime Text 2) to create my book “Markdown By Example” (http://markdownbyexample.com/).
Check out Leanpub if you intend to write a book in Markdown. I found it much easier than creating your own CSS stylesheet, especially since they also provide means of selling the ebook.
Also, they added some nice additions to the standard Markdown syntax, which can help.

Comments are closed.

Affiliate Disclosure: Make Tech Easier may earn commission on products purchased through our links, which supports the work we do for our readers.

Using the convert command, you can modify image files without having to open an image editor.

Most of us probably know how wonderful a tool Gimp is for editing images, but have you ever thought about manipulating image files on the command line? If not, let me introduce you to the convert command. It easily coverts files from one image format to another and allows you to perform many other image manipulation tasks, as well — and in a lot less time than it would take to make these changes uses desktop tools.

Let’s look at some simple examples of how you can make it work for you.

Converting files by image type

Coverting an image from one format to another is extremely easy with the convert command. Just use a convert command like the one in this example:

The arrow.png image should look the same as the original arrow.jpg file, but the file will have the specified file extension and be different in size. The convert command will use the file name you provide (in this case, arrow.png) to determine what type of file you want to end up with and reformat the image accordingly. If you doubt that the proper conversion could possibly happen as quickly and easily as it does, you can always use the od command to examine the beginning of each of your files.

Yes, convert did all that in a few seconds. The “JFIF” string displayed in the first file and “PNG” displayed in the other verify that the operation performed as instructed. These are the internal codes used to identify these two file formats.

Reformatting large groups of files

If you want to use the convert command to generate hundreds of .jpg files from hundreds of .png files (or vice versa), you can use a script like this. Depending on the number of files to be converted, it still will likely take only take a matter of seconds or possibly minutes to complete the task.

Note: Use sed ‘s/jpg/png/’ if you want to convert jpg files to .png format.

While I’m only showing .png and .jpg files in these examples, convert will work with other image types as well (e.g., .bmp, .gif and .tif).

Rotating and flipping

To rotate a file, you need to tell convert how many degrees to rotate it. The first of the two commands below will rotate an image 45 degrees in a clockward direction. The second uses the -flip option to turn it upside down.

Notice that in the first command we’re doing the rotation and the conversion to a .png file with a single command.

Keep in mind that a 180-degree rotate and a flip are not the same thing. The rotate command will create an image that looks like you’ve spun the original image around halfway. The flip will create mirror image of the original image.

Annotating

To add annotation to an image, use a command like this one, but make sure the font size works for your image. Note that you should select a text color that will show up on your image and that the size and position of the text must be specified.

The pointsize setting should depend on the size of your image file. The coordinates are relative to the upper left corner of the image. Experiment with the settings, and you’re sure to find some that work well for you.

Appending

In this command, we’re appending two images together to form a single image.

The original and flipped images are appended — one on top of the other.

Many other options .

The Convert – Linux man page says the command can “convert between image formats as well as resize an image, blur, crop, despeckle, dither, draw on, flip, join, re-sample, and much more” — and that’s no exaggeration. While the basic operations are extremely fast and easy to perform, the huge variety of image manipulation options is somewhat mind boggling. Install the program and experiment with some sample images, and you’ll slowly discover many options that you will enjoy using.

Wrap-up

Once you decide what kind of image manipulation you want to perform and work out the command that makes the intended changes for you, you can run through a set of files with surprising speed — much faster than making the same type of changes using Gimp. The convert command is a real winner for the “no artistic skills needed” kind of image manipulation.

I hope you enjoy working with the convert command. And now you know what I’d look like as a Simpson!

Sandra Henry-Stocker has been administering Unix systems for more than 30 years. She describes herself as “USL” (Unix as a second language) but remembers enough English to write books and buy groceries. She lives in the mountains in Virginia where, when not working with or writing about Unix, she’s chasing the bears away from her bird feeders.

Overiew

In this tutorial, you will learn how to base64 encode and decode from the command-line on Linux. You will also learn what base64 encoding is and why it should never be used to protect data from unauthorized access.

Base64 encoding and decoding data has many use cases. One being is ensuring data integrity when transferring data over the network, while another is storing Secrets in Kubernetes.

After reading this tutorial you will understand how to easily encode files or strings, and then decode them back.

How to base64 encode on Ubuntu, Debian, OSX, and Red Hat

If you are running popular linux distributions, such as Ubuntu, Debian, CentOS, or Red Hat, the base64 command-line tool is typically pre-installed. You should not have to perform any additional steps.

OSX also comes bundled with its own version of base64.

Why Base64 Encode Data

Transferring an ASCII file over the network can cause corruption if not decoded correctly. The reason is ASCII files are string converted to bytes, and when those bytes are decoded incorrectly back to ASCII your data becomes corrupt.

Base64 was introduced as a way to convert your ASCII data into arbitrary bytes, where they could then be transferred as bytes, and decoded correctly back to ASCII.

In short, base64 encoding ensures the integrity of our data when transferred over the network.

Base64 is not Encryption

Encoding files is not encryption and should never be used to secure sensitive data on disk. Rather it is a useful way of transferring or storing large data in the form of a string.

While it may obfuscate that actual data from should surfers, anyone who has access to base64 encoded data can easily decode it.

Base64 Encoding a String

To base64 encode string you can pipe an echo command into the base64 command-line tool. To ensure no extra, hidden characters are added use the -n flag.

Without the -n flag you may capture a hidden characters, like line returns or spaces, which will corrupt your base64 encoding.

Which will output the following

Base64 Encoding a File

To base64 encode a file

This will output a very long, base64 encoded string. You may want to write the stdout to file instead.

Decoding Strings

To decode with base64 you need to use the –decode flag. With encoded string, you can pipe an echo command into base64 as you did to encode it.

Using the example encoding shown above, let’s decode it back into its original form.

Provided your encoding was not corrupted the output should be your original string.

Decoding Files

To decode a file with contents that are base64 encoded, you simply provide the path of the file with the –decode flag.

As with encoding files, the output will be a very long string of the original file. You may want to output stdout directly to a file.

Conclusion

In this tutorial, you learned how to base64 encode files and strings. This something commonly done to transfer files in such a way that it remains