How to Manage Files and Directories in Bash

Image by Author | Midjourney & Canva

Understanding the Bash Shell

Bash, the Bourne-Again Shell, is a command-line interpreter that allows users to interact with an operating system by typing commands. It's commonly used in Unix-based systems like Linux and macOS and provides myriad tools for managing files and directories.

To start using bash, you will need to open the terminal:

• On Linux, look for the terminal application in your application menu.
• On macOS, use the Spotlight search (Cmd + Space) and type "Terminal."
• On Windows, you can use Git Bash or the Windows Subsystem for Linux (WSL).

Once you have the terminal open and at your disposal, we are ready to learn how to manage files and directories with bash. We start with some basic navigational commands, and then move on to managing directories and files.

pwd – Print Working Directory

The pwd command displays the current directory you are in. This is useful to confirm your location in the file system.

pwd

ls – List Directory Contents

The ls command lists the files and directories in the current directory. You can add options like -l for detailed information or -a to include hidden files.

ls  ls -l  ls -a

mkdir – Make Directories

Syntax: mkdir <directory_name>

Example: Create a directory named data

mkdir data

You can create multiple directories at once:

mkdir dir1 dir2 dir3

To create nested directories, use the -p option:

mkdir -p parent/child/grandchild

rmdir – Remove Directories

Syntax: rmdir <directory_name>

Example: Remove an empty directory named data:

rmdir data

Note that rmdir only works for empty directories. To remove non-empty directories, use rm -r.

cp – Copy Files and Directories

Syntax: cp <source> <destination>

Example: Copy a file named file.txt to the backup directory:

cp file.txt backup/

To copy multiple files:

cp file1.txt file2.txt backup/

To copy directories, use the -r (recursive) option:

cp -r dir1 backup/

mv – Move/Rename Files and Directories

Syntax: mv <source> <destination>

Example: Move a file named file.txt to the backup directory:

mv file.txt backup/

Rename file.txt to file_backup.txt:

mv file.txt file_backup.txt

The mv command can move files/directories and rename them.

rm – Remove Files and Directories

Syntax: rm <file_name>

Example: Remove a file named file.txt:

rm file.txt

To remove directories and their contents, use the -r (recursive) option:

rm -r dir1

For forced removal without prompts, add the -f (force) option:

rm -rf dir1

Practical Examples for Data Scientists

Creating a Project Directory Structure

Example: Create directories for a data science project

mkdir -p project/{data,scripts,results}

Organizing Data Files

Example: Move all .csv files to a data directory

mv *.csv data/

Cleaning Up Unnecessary Files

Example: Remove all .tmp files

rm *.tmp

Combining Commands

Using && to Chain Commands

Example: Create a directory and move files in one command

mkdir backup && mv *.csv backup/

Using Semicolons to Execute Sequentially

Example: List contents and then remove a file

ls; rm file.txt

Tips and Best Practices

Safety with rm

Always double-check paths before using rm to avoid accidental deletion.

Using Wildcards

Wildcards like * can match multiple files, making commands more efficient. For example, *.csv matches all CSV files.

Backup Important Files

Before performing bulk operations, create backups to prevent data loss.

Quick Reference

Here is a quick reference summary table, summarizing the syntax and use of cp, mv, rm, and mkdir.

Matthew Mayo (@mattmayo13) holds a master's degree in computer science and a graduate diploma in data mining. As managing editor of KDnuggets & Statology, and contributing editor at Machine Learning Mastery, Matthew aims to make complex data science concepts accessible. His professional interests include natural language processing, language models, machine learning algorithms, and exploring emerging AI. He is driven by a mission to democratize knowledge in the data science community. Matthew has been coding since he was 6 years old.

More On This Topic

• Stop Hard Coding in a Data Science Project – Use Config Files Instead
• 3 Ways to Process CSV Files in Python
• 5 Genuinely Useful Bash Scripts for Data Science
• Building Your First ETL Pipeline with Bash
• How to Navigate the Filesystem Using Bash
• Be prepared to manage the threat with an MS in Cybersecurity from…