Ant Execute plugin with grep problems

Ok, so it's not often I find myself dealing with shell scripts - it's just not what I do. But today I had to modify a build script to accept arguments and to search certain files for certain phrases. The argument stuff was fairly straight forward - just use getopts. That obviously isn't the most powerful option, but it's all I need.

Now for searching for a particular string, previously I chose sed, because I was replacing macros. But now, I just want to test for existence - does this file contain this string - for which sed seemed wrong. So I chose grep, a somewhat obvious option. I made the changes, tested it out by running the shell script directly, and things were cool. Enter Ant.

For the build proper, we use Ant. In this, we use the Execute plugin to run that shell script I just mentioned. Now this is where my problem started.

What I saw:

execute:
        [exec] Result: 1

What I expected:

execute:
        [exec] Found Macro, replacing it.
        [exec] Done
        [exec] Result: 1

Obviously something is amiss. "Done" should be printed no matter what, unless there is an error. But there was no error being printed. So I hunted down the problem line:

    gn=$(grep "['\"]{[[:alpha:]_][[:alnum:]_]*}" notice_tmp.js)

And I narrowed that down to just grep. I was seeing this strange behavior where as I changed the search string, sometimes the execution would finish and print "Done", and sometimes it didn't. I was fairly perplexed till I finally said the problem out loud to myself: "Sometimes it exits correctly, sometimes it doesn't". . . ...  'exit'! Eureka!

Ok so maybe people who deal with shell scripts a lot are like "no shit Sherlock" but I'm not, so it took me a while, especially with the problem being a few layers deep in commands. Anyways, it occurred to me that a call to exit inside grep must be bubbling up to Ant causing it to exit early. So when there IS a match found by grep, it doesn't exit and continues in my script, but when there ISN'T a match, it exits immediately and my script is just out of luck.

Fortunately, Google solved this for me rather quickly. Since an exit is bubbling up, just put another command on top of it to catch it:

    gn=$(grep "['\"]{[[:alpha:]_][[:alnum:]_]*}" notice_tmp.js | echo)

or, if that doesn't work out for you, another way:

    gn=$(echo $(egrep "['\"]{[[:alpha:]_][[:alnum:]_]*}" notice_tmp.js))

And there we go, echo catches the exit call, and my script keeps on running

Javascript beyond the basics - important stuff you should know

Presentation here

Firefox File Out (nsIOutputStream) details - ArrayBuffer, String, and Array

A few helpful things to remember:

Goal: Write out binary data stored in a ArrayBuffer (html5 typed array) to a file

What I learned:

If you open a file with FileUtils.openSafeFileOutputStream, then the only way to properly close it is with FileUtils.closeSafeFileOutputStream. This is something the docs forget to tell you but very important.

Typed Arrays have bounds checking on instantiation. If byteLength%BYTES_IN_TYPE!=0 then an exception is thrown.

String are written to memory sequentially. Array indexes are not. TypedArray index locations can't be accessed directly (reading them actually reads their wrapper object, or to be more precise, it seems if you run "var a = buffer[i], b = buffer[i+1];" you would find that &a is not related to &b).

Writing out in an OutputStream will convert to a string whatever object you pass into the buffer parameter!!! The result of such a call (toString) will be what is set as the location in memory of the buffer, assuming typeof buffer.toString() == "string". If you could spoof that, that would be wearrryy wuseful.

Ways to write to a file
1) NetUtil.asyncCopy
2) nsIFileOutputStream
3) nsIBinaryOutputStream <--- <3 some love


I tried and tried and tried some more, to find some Firefox service or component which could just write the binary out WITHOUT conversion to a string first. Couldn't find one. They ALL took in a nsIInputStream only. Which sucks because my data isn't an input stream, AND I can't just implement nsIInputStream in javascript due to "Native Only" method calls.


So, any way to have a ArrayBuffer backed InputStream?? That would be nice. But no soup for me, it seemed.


I was just trying to avoid implementing my own buffered writer.  Anyways, given that FileOutputStream only accepts string input, and NetUtil.asyncCopy only takes a legit input stream, I've no choice but to convert to a string manually **(or so I thought).

So, given the items I learned above, this is what I had to do:



But but, right after I finished with this, it occurred to me that for every InputStream, there must be an OutputStream. Since there was a BinaryInputStream, I did a search and indeed, I found BinaryOutputStream. I'd no particular reason to believe that it would behave differently than FileOutputStream, but just to make sure, I tested using a Typed Array as the output. Low and behold, it worked!! Seems BOS runs through the index values of an object. I would wager that it actually creates an Iterator and runs through that, but I didn't test that.

So once I knew about BOS, my code simplified to using just service calls, which is nice.