I’ve realised today that the following Open-Source Generic Command-Line tools (developed in Python) are great “lifehacks”. Without further ado:
Never lose a command:
The idea is that bashhub will store all the commands you use on any connected device, and you can use
bh <keyword> to find back how you were exactly supposed to use that exotic tar command. Now, without using google, you can obtain a valid tar command within 10 seconds. Use
bh tar to find commands previously including
After installation, you will have
bashhub as command available to handle bashhub itself, while
bh can be used to query.
bashhub status prints:
Total Commands: 15838 Total Sessions: 859
I also love the builtin update capability
bashhub update which updates to the latest version.
Other than query for tar, you can ironically also use
bh on itself (
bh bh). This is an example of my last searches using bashhub:
bh new bh solc bh geth bh /electron bh bup bh yagmail bh restore bh ssh
Control-r (reverse-search) on steroids!
Smarter password manager:
Recently I started making more complex passwords, resulting in having to restore my passwords more and more often. Counter-productive. It was time for a password manager, but I do not like external password managers. I do not like any organisation to hold all my passwords.
We do want our passwords to be strong, recoverable and easily available. visionary is building towards that (API will be coming for “easily available” password generation).
visionary deterministically generates the same password for the same source by utilizing the Scrypt algorithm. The password depends only on the strength of the algorithm (can be chosen), the master password and a keyword.
For example, to generate a password for facebook, you might use the keyword “facebook.com”. If the keyword would be difficult (facebook123catsname), it’d be just like having to enter another password; beating the purpose. Since most people would use a similar keyword flavor, we cannot really see this keyword as an extra sense of security. However, it serves very well against brute force cross-domain password attacks. Using only a single character different between keywords create completely different 32 character passwords. Thus, when someone would obtain your password on one website, it cannot be related to a password on another website. So, in the end, we’re left with a password manager that actually solely depends on the master password, and I’m fine with that. The only way people are going to get my strong master password is when my PC has been locally compromised. Even in the wrong hands, since this is not a big password manager yet, “obscurity” is another layer of defense :-)
visionary’s code for illustrative purpose:
def generate(master_password, keyword, cost=2048, oLen=32): hashed = pyscrypt.hash ( password = master_password.encode('utf-8'), salt = keyword.encode('utf-8'), N = cost, r = 1, p = 1, dkLen = 32 ) return codecs.encode(hashed, 'hex').decode('utf-8')[0:oLen]
Anyway, for more info and usage, you should read the README on the github page. To get the basic idea:
pip install visionarypm for setup. Then run
vpm to start. First choose Scrypt’s strength. Then enter a master password and a key. The next time we have to get the password we just run the
vpm, enter a master pasword and keyword, and end up with the same result as before.
Even though most my projects run through Git, I still feel much safer when my precious scripts / projects not on git are still recoverable / accessible.
Luckily, I work simply from a single project folder (
~/projects). Everything outside of that is not so interesting to me, though when Arch Linux becomes my main OS, I would consider using it for the
/etc/ (config) folder. It is easy to include multiple paths regardless.
Earlier on I’ve used both Dropbox and GDrive, but they lack an “ignore” file, or are too much focused on a single folder. I cringe whenever I see GDrive upload endlessly after I did another
npm install, or when it’s uploading some sklearn pickle.
After considering Dropbox, GDrive, Syncthing and BTSync, I’m glad to have stumbled upon
bup is very efficient at using the good parts of git. It also handles binary data and large files by splitting them up. So, it does incremental updates really well, and we get versioning added in! For a complete talk by Zoran Zaric (the creator), watch this video.
I have a script that I will run with cron every hour in the evening. I find the
bup documentation lacking, but bits and pieces are certainly available on the internet. A great example script is available here for backing up to External Disk and/or Raspberry Pi.
You could of course also just backup to another PC in the network, or to some cloud storage. The example below intends to backup through SSH.
Note that you have to install
bup on both machines, and have run
bup init on both machines.
Specifically, I used:
BUP_DIR=/home/pascal/MACBACKUP bup init
to initialize the
bup directory in
MACBACKUP – the place where I will want to backup my Mac’s files on the remote machine.
Locally, I just used
bup init (defaulting to creating .bup in
Afterwards, it is then possible to start backing up. Here are the commands I use in my cron script:
/usr/local/bin/bup index ~/projects --exclude-rx \.git --exclude-rx node_modules --exclude-rx \.hg --exclude-rx \.tox /usr/local/bin/bup save -r archlan:MACBACKUP -n macbackup ~/projects
bup is prefixed by
/usr/local/bin/ since otherwise
bup would not be found for the cron user.
bup index will index the differences, where:
~/projects # contains all the files to be included --exclude-rx ... # will exlude any files where the regex matches
bup save will then actually transfer the data:
-r # signifying using remote through ssh archlan # archlan is my ssh config, which expands to email@example.com : # used in the ssh syntax: <username>@<remote-ip>:<folder-path> MACBACKUP # the folder /home/pascal/MACBACKUP on my secondary machine
It also does great compression.
Including git/node_modules: 24G Excluded: 19G Compressed into 7G
So now I have a great backup, which can be stored where we want, versioning, and efficient updates. Oh yea, you will want to restore. Use:
bup restore -C ~/destination_folder macbackup/latest/
Hopefully you found these tools as useful as I have. Feel free to leave a comment below.