Bash Tutorial: Getting Help

Getting Help

 Let’s face it: you need help.  A lot of help.  Your mom has known this for quite a while now, but you are just beginning to realize it.  So, let’s explore what you can do about it when she is not around.

 Let Me Google That For You / RTFM

Before you ask your pressing question to the world on Facebook, mailing lists, online forums, Twitter, and the corporate global e-mail address, first pause and do nothing.  Before you get down on your knees and pray for the answer to your pressing question (e.g., “What does the echo command do?”), reconsider.  Type your query into Google first.  You may be astonished to find out that your question has more often than not already been answered.  Others have been confounded by the very same mystery you are stuck on.

 Here are some websites where your question has already been answered:

http://stackoverflow.com/
http://www.tldp.org/
http://www.gnu.org/software/bash/manual/bashref.html
http://www.linfo.org/main_index.html
http://mywiki.wooledge.org/BashGuide

“That’s all well and good, Jeff,” you say, “but I’d have to lift my hand off my keyboard, move it to my mouse, launch a web browser, go to Google, type in a query, and hit enter to do that.  I have enough hassle in my increasingly short life.”  Wow, I’m surprised you’ve read this far given your fierce work ethic and indomitable spirit.  Fear not, if you have Launchy installed (and Firefox as your default browser), you can do it all from the keyboard.  Ah, all the fools and their pathetic touch screens!  Type <Alt + Space>google<Tab>Bash<Enter>.  Now <Tab><Enter>.  Bam!  You are on the Bash Wikipedia page without the hassle of using your mouse.

If you can’t figure out how to install Launchy, TRY GOOGLING IT!  If you can’t figure out how to configure it, RTFM!

 When the Internet Is Broken

If the unthinkable occurs, you can still get help from your trusty computer.  Fortunately, not everything has been assimilated by The Cloud yet.

$ man bash
$ info bash
$ help <built-in>
$ <non-built-in command> –help

When I use <foo>, I don’t mean type <foo> literally.  I mean “replace <foo> with something that is a foo”.  In this case, type something that is a Bash built-in command.  If you just type ‘help’ by itself, you will see a list of Bash’s built-in commands.

A built-in command is a command that Bash handles by itself.  Non-built-in commands are compiled binary executables or other scripts that exist as separate files on your computer.  Often they will support a –help command-line option.  Types of commands, command-line options, command-line arguments and a bunch of other related topics will be discussed in more detail later.

 Helping Yourself

Well, now you’ve learned a bunch of useful stuff, and what did you do with it?  Nothing.  You did not write it down.  You will not remember it.  Nice work, Einstein!  As Gandalf would say, “Fool of a Took.  Throw yourself in next time, and rid us of your stupidity. “  May I suggest a different approach: write down your hard-won knowledge.  When God talked to Moses, was that it?  Did he wander off and think, “Moses is a sharp guy–he can remember 10 things.”  No!  He wrote that stuff down on a couple stone tablets (the very ancient equivalent of CD-Rs).  You also have the ability to write down important things.  Suppose you learn something interesting about the echo command.  Maybe you could write that information in a file like ~/Dropbox/Notes/Linux/Commands/echo.txt.  Maybe that file is in Dropbox.  Maybe that file is indexed by Launchy so you can type <Alt + Space>echo.txt<Enter> to get to it really, really fast.  You may end up with a focused, quickly accessible database of knowledge pertaining to the things most relevant in your life.  Perhaps it is even under revision control on a machine that is backed up regularly.  Then, it really is like stone.

Another good way to help yourself is to write what I call ‘hello projects’.  These are simple one-file projects focused on the basics of a topic in some language.   Let’s say in the near future, you are learning about the basics of for loops in Bash.  You suddenly have the brilliant idea to make a hello project: ~/Dropbox/Projects/Hello/Bash/hello_for.sh.  Now, whenever you forget about the basics of for loops in Bash, you can look at a very focused simple piece of actual working code.

Over time, instead of asking people annoying questions, you will waste less of their time by answering them yourself.  I salute you for your contribution to society.

 If All Else Fails

If you have completely failed in your quest for knowledge, humble yourself before your fellow humans and ask a question.  While you will suffer humiliation for a second, you will not remain in ignorance for an eternity.  Asking random people at the mall will not work.  People over 60 may give you a blank look.  Small children (and many grownups) may start crying.  Try instead a tech-oriented co-worker, the mailing list of a local Linux User’s Group, or a relevant online form.  If you are still not getting results, try one of these: http://www.amazon.com/Mattel-30188-Magic-8-Ball/dp/B00001ZWV7/.  Or just ask your mom (at least if you are Dilbert).

Python 3: Keywords

I’m learning Python 3 at the moment.  I’ve done some Python 2 before.  Working through Learn Python 3 the Hard Way is interesting.  Lots of making little text adventures.  Interesting to me because I played a lot of them in their short golden era.

Anyway, the author has you periodically enumerate all the interesting symbols you’ve seen in the language and briefly state what you think they mean.  I’m going to do that just for Python 3 keywords here.

and
Logical and with shortcut evaluation.
True and False == False

as
Rename symbols when imported; also used with “with as”.
from sys import argv as clas # Command-line args.
with X as Y: pass

assert
Assert that a boolean expression is true.
assert False, "Error!"

break
Exit from innermost loop.

while True:
    if 1 == 1: break

class
Define a class.
class SoftwareDeveloper(Superhuman): pass

continue
Start innermost loop again (skip any remaining instructions in current iteration).

while True:
    if 1 == 1: continue

def
Define a function.
def myfunction(arg): pass

del
Delete an item from a dictionary.
del d['property']

elif
Start an else if block.
if : pass
elif : pass

else
Start an else block.
if : pass
else: pass

except
Catch an exception.
try: pass
except ValueError, e: print(e)

False
Boolean false literal.
False

finally
Execute this block even if exceptions were raised.

try: pass
except: pass
finally: pass

for
Loop over an iterable.
for element in list: pass

from
Import specific symbols from a module into the current namespace.
from sys import argv

global
Declare that the variable you mention is a global, not a local.
global some_otherwise_local_variable

if
Start an if block.
if True: pass

import
Import a module as a whole.
import sys

in
Check if an element is contained in something else or form for loops.
if choice in "Yy": pass
for element in list: pass

is
Tests equality somewhat like ==. Is this like Java equals()?
1 is 1 == True

lambda
Create a short anonymous function.

squared = lambda y: y ** y
value = squared(4)

not
Logical not.
if not 1 == 1: pass

or
Logical or.
True or False == True

None
Literal value for NoneType.
o = None

nonlocal
This is not local, but it’s not quite global either.
nonlocal var_in_some_enclosing_nonglobal_nonlocal_scope

pass
Do nothing (used to create empty block).
def noop(): pass

print
Okay, this is not a keyword. Builtin to print string and trailing newline.
print("Hello, Python 3!")

raise
Raise an exception.
raise ValueError("No")

return
Return a value (can be a tuple) from a function.
return result

True
Boolean true literal.
True

try
Start an exception handler block.
try: pass

while
Start a while loop.
while True: pass

with
Do a block with an expression as a variable.
with X as Y: pass

yield
Pause and return to caller. Used in generators.
def x(): yield y; x().next()

Bash Tutorial: Basics

Foreword

I wrote a few Bash Tutorial articles back in 2014.  I am now reposting them here.

It is very hard to write a very basic tutorial on anything having to do with software development.  Anyone that has developed software for years is steeped in a massive amount of background information that is impossible to completely unassume.  Now that I’ve washed my hands of all responsibility for failing to teach you anything (and presumptuously done so using the nonword “unassume” which I only assumed existed when I should have been more unassuming), I shall proceed.

What is Bash?

Bash is the “Bourne Again Shell”.  It is the successor to the “Bourne Shell”.  AFAIK, it has nothing to do with The Bourne Identity series of movies.  Nor is it a religious movement.

So what is a shell?  A shell is an enclosure that protects the thing inside from the stuff outside.  In this case, you are the stuff outside trying to destroy the inner workings of your computer.  By this definition, shells have not been very successful.  You’ll end up destroying your computer despite its efforts to prevent this.  More specifically, a shell is a program that lets you interactively issue commands to your computer.  Without a shell, you’d have to write lower-level programs in languages like C or C++ that make calls to system libraries whenever you wanted to do anything interesting on the computer.  That’s no fun, so we software developers have spend thousands upon thousands of man hours building layers upon layers of software to make it easier for you to talk to your computer.  And all you end up doing is posting a picture of your lunch on Facebook while watching a funny cat video on YouTube.  You’re welcome?  Anyway, a typical computer system has these layers:

hardware
dust
NSA-infected or Chinese-infected firmware
kernel and drivers
system libraries
Facebook, spyware, adware, and anything from Norton
shell
screen
the sticky notes comprising your enterprise project management system
your password, ‘password123’, written on the back of said sticky note
keyboard
dusty crumbs
a cat napping on the keyboard
mouse
you (at least the 15 minutes of the day when not on break, at lunch, or in a meeting)

You can consider a graphical desktop environment as a visual shell.  Bash is a text-based shell, that is, a command-line shell.  You type in text and get text back as output, along with other side effects like deleting all your files.

Why Should I Care About Bash?

Unless you are or are trying to be a programmer, sysadmin, or something similarly odd, you probably won’t care.  You probably stopped reading once you hit the words “computer” or “definition” and started reading my PowerPoint for Winners series.  You believe all necessary computing problems have been solved by Amazon Prime Pantry, the popcorn button on your microwave, and Netflix.

If not, a shell lets you do the same kinds of things you can do in a normal desktop environment: create files, move files, delete files, search for files, manage directories, create links/shortcuts, launch programs, terminate programs, etc.  For certain activities, a command-line shell like Bash tends to be (far) more efficient and powerful than a visual environment.  Also, Bash is the standard shell on Linux and is available on Windows machines via Cygwin.  Since OS X is Unix-based, Bash is available there too.  Basically, whatever environment you are working in, you’ll likely have the phenomenal cosmic power of Bash available.  So, why not use it?

Another advantage of a textual shell, is that it is far more scriptable than a visual shell.  Sure, you can use AutoHotkey (AHK) to do cool stuff on the desktop, but it’s more of a pain.  The same commands that you type into the shell interactively can be put directly in a file and used as a script.  This lets you automate tedious things that would take you hours to do by hand.  Instead of wasting your time on all that tedious stuff, you’ll get to waste your time on trying to figure out why your scripts aren’t working!  I’ll give you a hint: you either have whitespace where it shouldn’t be or don’t have it where it should be.  But seriously, hopefully your automation efforts will produce a net gain.  I have seen people’s entire jobs replaced by relatively simple scripts.  You want to be the person writing the script, not getting replaced by it.  The robots are coming, my friends.  In any case, it will be more fun, and more useful to put “Bash scripting” on your resume than “google”, “double click”, and “drag and drop” in the skills section.  Some people have not sufficiently acquired the “google” skill yet though.  For sheer resume points, knowing Python is more valuable.  However, shell like Bash still shine in their interactive/tactical capabilities more than Python (which is a pure scripting language not intended to be used as an interactive shell).  You can script in Bash, but it is more of an awkward side note.  A very direct, portable, widely-used awkward side note, however.  Really, you want to know all three: Bash, Python, AHK.  There’s very little tactical programming that cannot be accomplished with those three.

How Do I Get Bash?

This tutorial is written from a Cygwin/Windows 8 perspective.  Cygwin is a Unix-like environment that runs on Windows.  If you install Cygwin, you will have Bash.  It will be the default shell that runs when you run Cygwin.  If you are on a Linux machine, you likely just have to open a terminal program and you’ll be running an interactive Bash shell.

 So, download and install Cygwin.  I do an everything-and-the-kitchen-sink installation.  This will likely take hours to complete, so be patient.  You could do a full Ubuntu install on a VM faster.

http://cygwin.com/install.html

 You should now have a shortcut on your Windows desktop called “Cygwin64 Terminal” or something similar.  Launch that, and you should now be in an interactive Bash session.  A session is just a series of communications between two or more entities that follows some sort of protocol (communication rules).  So, you and the computer are now communicating by the rules of Bash.  Be careful what you ask it to do, since it may well do it.  Contrast this with your children, coworkers, neighbors, pets, elected officials, and Amazon Echo.  Okay, I apologize Alexa–you sometimes do what I ask you to.

How Do I Know I Am Using Bash?

I’ll use a $ at the beginning of a sentence to indicate you should type a command in at the shell prompt.  After the command, I’ll list the expected output.  This is a common convention.  So, do this:

 $ echo $BASH_VERSION
4.1.11(2)-release

 If you see something radically different, maybe your login shell is not set to be Bash.

$ echo $SHELL
/bin/bash

 $ cat /etc/passwd | grep $USER | cut -f 7 -d':'
/bin/bash

Don’t worry if you do not understand the commands above.  I don’t actually understand them either, but that hasn’t stop me from writing this or finding gainful employment.  Just stay positive and exude confidence.  If enjoy having a basis for your confidence, read the man pages (man => manual) for the various commands.  Press ‘q’ to exit back to the shell from reading the man pages.

$ man man
$ man echo
$ man cat
$ man grep
$ man cut

If your shell is not bash, you should be able to start bash anyway (below).  Note that when I say ‘bash’ I mean the actual executing shell process on your computer.  When I say ‘Bash’, I mean the concept or language of Bash.  And when I say ‘bAsH’, it’s just a typo.  And when I put the comma outside the quotation marks, I do it intentionally.  Ask your system administrator to change your default shell to bash if desired.

$ bash

How Do I Escape from the Horrors of Bash?

You can’t.  It’s probably running on the server on every website you connect to.  Watching you.  Waiting.  Helping to serve up irrelevant ads for things you already bought.  If you’d like to try, once you’ve developed a case of carpal tunnel from all the other typing, type ‘exit’ to exit to the relative safety of Windows.

$ exit

What’s Next?

Read the Bash Guide for Beginners.  I intend to elaborate on topics in that book as I read through it myself.  Or at least I intended to in 2014 before an endless barrage of other work derailed those plans.

Here’s some historical background on the command line that you may enjoy: http://www.cryptonomicon.com/command.zip.

Keep bashing away.  It takes years to become very proficient on the command line.  It is worth it if you intend to become a highly skilled software professional.

 

Scrum: The Role of the Development Team

There are three roles in Scrum: Product Owner, Scrum Master, and Development Team.  Collectively, they form a Scrum Team.  I recently became a certified Professional Scrum Developer (PSD I).  I support Scrum because I have seen the shockingly horrible results of being forced to violate just about all of its tenants simultaneously.  I understand by the pain of experience.  Given that, I feel like it is a good time to summarize what I know about the Development Team role in Scrum in order to help other understand it more fully.

As a side note, RIP Tom Petty.  Great musician!  “I Won’t Back Down” either.

Development Team

  • Each person on the development team has only the title Developer.
    • This promotes respect (a Scrum value) by countering the ego generated by pretentious titles.
    • This promotes cross-functional teams by not assigning people specific roles like Architect, Web Developer, Database Administrator, or QA Tester.
    • You may think this is a trifle, but when pay is linked to title and people start asserting titles for which the rest of the team does not feel they are qualified, serious division and political strife occurs.
  • Is self-organizing (has autonomy).
    • Autonomy, mastery, and purpose are prime motivators above the carrot/stick level.
    • This respects the immense talent that software development professionals have.  Micromanaging such talent is like telling your doctor how to do surgery.  It’s absurdly foolish!  Yet it happens every day in poorly managed organizations.
  • Does not include the Product Owner and the Scrum Master.
    • Unless they are acting in hybrid roles (which is a very bad idea).
  • Ideally consists of 3 to 9 developers.  I believe the sweet spot is 7.
    • Less than that, and it is almost impossible to have the cross-functional skills and raw firepower necessary to build a meaningful increment of software within a sprint.
    • More than that, and subteams naturally form which undermines team unity.
    • More than that, and exponential communication overhead becomes too costly.
  • Controls the sprint backlog.  Decides how much to pull from the product backlog each sprint.
    • No one (not even the meddling, micromanaging CEO) can push work onto the sprint backlog.  Well, at least they don’t in an organization that is actually agile (rather than just claiming to be in company e-mails riddled with other lies and false promises).
    • Neither the Product Owner nor the Scrum Master can push product backlog items into the sprint backlog.  Nor can they tell the Development Team how to translate those requirements into development tasks.
  • Performs all estimates.
    • No one (not even the horrible CEO pulling fantasy deadlines out of his ass) can estimate the work to be done.  Even then, estimates are not contractual commitments to whip the peons with.  It is offensive, disrespectful, and completely ignorant for upper management to employ such tactics.
  • Is dedicated to a single project.
    • One of the Scrum principles is commitment.  If every person on your team is assigned to multiple projects, there is no commitment to any specific project.  This won’t stop the unethical, manipulative, evil CEO from beating you with the 1 project that failed even when 4 succeeded.  Being evil, he was setting you up as a scapegoat from the beginning.
  • Is collectively responsible for all development work needed to create the next software increment.
    • While one person may be the lead on a task, the team is still responsible.  This prevents team members from taking the attitude of “Well, my work is done, so I won’t be blamed if the project fails” and “I only have to not be in last place when being chased by a tiger.”
  • Collectively owns the code.
    • Man, I have seen some people get angry when someone messes up “their” code.  On my first real job, I accidentally messed up some esoteric bit in some Java code while trying to fix it and got banished to the IT room for a year.  Just for changing one line of code!  A trivial, easily-corrected problem that did not even escape into production.  Complete alpha-developer, ultra-territorial, ego-fueled, disrespectful bullshit.
  • Is cross-functional.
    • If the Development Team does not collectively have sufficient skills to produce a working increment of software, it can’t do Scrum.  This explains the hype surrounding “full stack” developers lately.
    • Skill redundancy protects against single-person bottlenecks.
      • You don’t want to be dead in the water just because your only firmware developer is on extended, indefinite leave (due to being death-marched by the CEO over Christmas).
      • Everyone needs to be able to fix the damn build.
    • In software development, the cost of context switching and context hand-offs is EXTREMELY HIGH!  The more vertical end-to-end work a single person or team can do, the better.
      • Think of it this way.  Your 400m relay team consists of Usain Bolt clones.  However, the cost of passing the baton is 1 minute.  If I, with zero athletic ability, run alone against your superstar 10x team, I will crush you into the dust every time.
      • The moral: Heroics and alleged 10x superstars do not scale in software development.  You are better off with a well-managed team of solid developers that have breadth of skill working at a sustainable pace.
  • Works at a pace that is sustainable forever.
    • Death marches are always a sign of bad management.  Always.  It’s better to 20-mile march as per Great by Choice, one of the many enlightening books I have read that helped shape my software development philosophy.
  • Has respect for each other.
    • Without respect, trust will erode which leads to transparency eroding which leads to action on false information (lack of awareness) which leads to failure.  This seems like something Yoda might say.
    • If a team member is really lacking in skills and not improving, has no commitment to following your software development practice, or has been consistently disrespectful to others, fire them immediately.  Trust me: Do not wait a single day.  It does not matter what technical skills they have.  Letting this kind of situation fester destroys the morale of the rest of the team.
  • Self-reports progress daily by burn down charts or some other publicly visible mechanism.
    • At my last job, my brother actually rigged automatic generation of sprint progress by counting checked-off “grains of sand” subtasks put into each GitHub issue.  It was great!   Before that, we had a project manager with no software experiences (our CEO justified this by the obtuse assertion that “there is no difference in doing project management for building a bridge vs. creating software”) intrusively asking us to guess percent complete for everything.  This was stupid–meaningless numbers were being reported and then altered by a person who did not understand any of the tasks.  After generating accurate progress metrics,  upper management stupidly decided they were not accurate (time proved that they totally were) and ignored them (mainly because they proved that their deadlines pulled out of thin air were complete fantasy).
    • Empirical process control is based on transparency, inspection, and adaptation.  The example above shows that transparency alone is useless if the information provided is ignored or actively suppressed.  Unethical, dishonest, incompetent CEOs try to cover up the truth so that they can lie to customers instead of working with them to create real value (merely extracting revenue via lies without creating real value is criminal–it’s called fraud).
  • Should be collocated in the same physical room if possible.
    • Yes, multinational work is more and more a reality these days.  Still, it is horribly ineffective and should be avoided whenever possible.
    • Since in-person, face-to-face communication is the ultimate (as Agile asserts), telecommuting should be used sparingly.
  • Holds the daily scrum.
    • Neither the Product Owner nor the Scrum Master is required to be here.  Nor do they run it.  Nor is the Development Team status reporting to them or anyone else in this meeting.
    • This is supposed to be a short (15 minutes maximum) meeting mainly to coordinate work and identify impediments.  The point is to inspect and adapt the daily work items to increase the odds of accomplishing the sprint goal.
    • This is not supposed to be a posturing meeting where people blow smoke or try to appear busier than others to justify their existence.
    • This is not the time for deep problem solving and lengthy technical tangents.
    • When working with remotely split teams, some virtual water cooler chat needs to happen to reduce the alienation of the remote team members.
    • This should be held at the same time and place every day to eliminate coordination costs.