BFG Cleaner

Use case: To remove sensitive data from a Git Repository hosted on a remote git service such as GitHub

References

Removing Specific Strings from History

Steps:

First clone a fresh copy of your repo, using the –mirror flag:

$ git clone --mirror git://example.com/my-repo.git

Download <bfg>.jar (BFG Cleaner) from: bfg-repo-cleaner
Create a .txt file named passwords
Move bfg-1.13.0.jar and passwords.txt to the folder containing the repository’s root folder.

Edit passwords.txt to remove specific text you wish to locate and remove from the repository.

 '23.23.aaaaa' # Replace literal string 'PASSWORD1' with '***REMOVED***' (default)
 service.php # Replace literal string 'PASSWORD1' with '***REMOVED***' (default)
 user=aps # Replace literal string 'PASSWORD1' with '***REMOVED***' (default)
 key=aaaaaa # Replace literal string 'PASSWORD1' with '***REMOVED***' (default)

 #Remove the string below
 "http://23.23./asdfgswr"

In the root of the file run the following to replace strings in files:

$ java -jar bfg-1.13.0.jar --replace-text passwords.txt my-repo.git

To delete files from the repository history

$ bfg --delete-files YOUR-FILE-WITH-SENSITIVE-DATA my-repo.git
When ready run the following commands:

$ cd my-repo.git $ git reflog expire --expire=now --all && git gc --prune=now --aggressive
Push the project to GitHub (or any other Git saving service) with the following command to rewrite history on all branches:

$ git push origin --force --all

Finally, once you’re happy with the updated state of your repo, push it back up (note that because your clone command used the –mirror flag, this push will update all refs on your remote server):

$ git push
You will notice you cannot find the removed or replaced strings in history.

Note:

Using BFG cleaner does not remove the secret from the latest commit.
You cannot update the history of pull requests on GitHub. To remove secrets from a pull request we have to make a request to GitHub to remove the sensitive information.
Contacting GitHub to remove sensitive information:
- https://support.github.com/request
  - https://docs.github.com/en/github/site-policy/submitting-content-removal-requests
    - https://docs.github.com/en/github/site-policy/github-private-information-removal-policy

Delete all local repositories people may have. BFG Creates new commits of existing commits to clean up. So, all local projects should be deleted.

Reference: https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository

Removing a Folder from History

Steps:

Remove the files from git and push.
Download <bfg>.jar (BFG Cleaner) from: bfg-repo-cleaner
Move the downlaoded <bfg>.jar (BFG Cleaner) into the git repository you wish to modify.
Open a terminal to the repository directory

Demo:

Opening a terminal to repository directory on windows:

Run the following:

 java -jar bfg-1.13.0.jar --delete-folders "some_stupid_folder_name" some-big-repo.git

Demo:

Powershell:

 PS C:\Users\steph\Documents\GitHub\server-personal> java -jar bfg-1.13.0.jar --delete-folders "media"

 Using repo : C:\Users\steph\Documents\GitHub\server-personal\.git

 Found 104 objects to protect
 Found 6 commit-pointing refs : HEAD, refs/heads/dev, refs/heads/master, ...

 Protected commits
 -----------------

 These are your protected commits, and so their contents will NOT be altered:

 * commit ddafc8f6 (protected by 'HEAD')

 Cleaning
 --------

 Found 107 commits
 Cleaning commits:       100% (107/107)
 Cleaning commits completed in 860 ms.

 Updating 5 Refs
 ---------------

         Ref                          Before     After
         ------------------------------------------------
         refs/heads/dev             | ddafc8f6 | 83471107
         refs/heads/master          | a9769709 | 05f0de7a
         refs/remotes/origin/dev    | ddafc8f6 | 83471107
         refs/remotes/origin/master | a9769709 | 05f0de7a
         refs/stash                 | 0ee0f40f | d292401a

 Updating references:    100% (5/5)
 ...Ref update completed in 44 ms.

 Commit Tree-Dirt History
 ------------------------

         Earliest                                              Latest
         |                                                          |
         ..............DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDm

         D = dirty commits (file tree fixed)
         m = modified commits (commit message or parents changed)
         . = clean commits (no changes to file tree)

                                 Before     After
         -------------------------------------------
         First modified commit | c9cd2c95 | 57353fce
         Last dirty commit     | a2f6b4f1 | 7e308b32


 In total, 161 object ids were changed. Full details are logged here:

         C:\Users\steph\Documents\GitHub\server-personal.bfg-report\2020-07-07\00-41-00

 BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive


 --
 You can rewrite history in Git - don't let Trump do it for real!
 Trump's administration has lied consistently, to make people give up on ever
 being told the truth. Don't give up: https://www.aclu.org/
 --


 PS C:\Users\steph\Documents\GitHub\server-personal>

Remove bfg.jar and passwords.txt from the root folder or add them to the .gitignore
Git push the changes to the remote repository (GitHub or any other Git saving service) with the following command to rewrite history on all branches:

$ git push origin --force --all
You will notice you cannot find the folder in history.

Journal

2018.10.31 created BFG Cleaner README.md and passwords.txt file. This was a Google Doc I made
2019.04.19 Created How to use BFG Cleaner & How to delete a commit from history. This was a Google Doc I made
2020.07.03 Updates syntax
2020.07.07 Added how to remove a folder from history
2021-07-24
- Updated BFG Cleaner information