JavaScript as a serious programming language?

Unbeknownst to many folks, even those developing websites, JavaScript libraries are hot right now. One reason is that they're being used on both the frontend (browser) as well as the server via Node.js. And so Drupal 8 includes two new JavaScript libraries, beyond the mainstay of jQuery, which are popular because they rewrite how we use JavaScript, much like Drupal changes how we develop PHP software. They are the Backbone.js and Underscore.js JavaScript frameworks. Since they're in core, Drupal itself can use them in its own JavaScript code. However, as far as I can tell nothing in core, besides Backbone, is using Underscore yet.

Backbone.js (http://backbonejs.org/) gives structure to web applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing API over a RESTful JSON interface.

The cool thing about Backbone is it splits the abstract representation from DOM events and HTML rendering in your JavaScript code. This is especially applicable to web apps which constantly get new data via AJAX from a REST server and then update the browser in-page rather than reloading a new URL and rendering the page's html on the server - for example via Drupal and it's normal theme/template system.

If you think of modern web applications like GMail or Trello (which is built on Backbone) they work mostly by not reloading the page but rather with each click of the mouse pulling in data from the server then rendering it somehow in the current page. Backbone is how Rdio, Hulu, and Pitchfork play music and videos continuously across "page loads". This can also be happening in the background independent of user activity to update content or display status messages or maybe show new email.

AJAX vs AHAH

In Drupal, you'll see a lot of references to AHAH. AHAH is basically AJAX except that it's honest about the X, which in AJAX stands for XML, but in Drupal using AHAH it's actual HTML which can be inserted into a page. Or it could be JavaScriptON that's parsed and, using some custom JavaScript logic, turned into HTML by manipulating the DOM.

Without something like Backbone, this is how AJAXy applications are written in Drupal. We add in #ahah hooks that form elements and their widgets can use (when JavaScript is available.) But the login on the frontend is completely custom and usually is jQuery $.ajax callbacks that use jQuery selectors to find and replace DOM elements manually.

Backbone.js gives us a framework to represent things we want to update on the frontend in a structured way, with clear separation between the model and rendering. The model language directly maps to the MVC of Ruby on Rails. Then by updating the model, which is all the logic we really want, the visual representation is updated for us. Backbone can call Underscore templates, Mustache.js, or directly insert into the DOM server-side rendered (the old way we just mentioned) HTML.

So now our AHAH callbacks can return JSON that's pure data, and allow Backbone to handle rendering the HTML.

So what about Underscore.js?

Backbone itself depends on Underscore, but Underscore by itself is also useful. For example, see how Conway's Game of Life can be implemented in JavaScript without if-statements or loops (for, for-in, while) by using Underscore.js and by programming in a more functional style, with functionality similar to Ruby or Python. jQuery, long since included in Drupal core, also offers some functions like $.map or $.each but Underscore is much more complete. Underscore is a JavaScript utility library, not for writing any specific kinds of apps but for making all of your JavaScript code potentially cleaner, simpler, shorter.

Some _ functions:
_.each
map
reduce
find
filter
every
pluck

Underscore also has some array and object helper functions, deep === comparison (isEqual), etc. See it all at http://documentcloud.github.com/underscore/

You can play around with Underscore.js right now by writing up some code at http://jsfiddle.net/

As I blogged recently, Drupal 8 core now has two new JavaScript libraries: Underscore.js and Backbone.js. But, like jQuery, these libraries or frameworks can also be extended with JavaScript modules that integrate or are based on them.

Backbone has a lot of projects based on it, whereas Underscore has relatively few, besides Backbone itself.

The main Backbone page lists a lot of Backbone resources: https://github.com/documentcloud/backbone/wiki/Extensions,-Plugins,-Resources

But here are some others:

http://kmalakoff.github.com/knockback/ - Knockback.js brings Knockout.js magic to Backbone.js

https://github.com/lukasolson/Backbone-Super - A convenient super method for the popular JavaScript library, Backbone.js.

http://chaijs.com/plugins/chai-backbone - Chai Assertion Library extends itself for Backbone support: chai-backbone is an extension to the chai assertion library that provides a set of backbone specific assertions.

Underscore plugins

Really, not nearly as much directly based on Underscore.js as on Backbone. But here are a few:

underscore-awesomer.js

https://github.com/kmalakoff/underscore-awesomer - By the author of Knockback, underscore-awesomer.js purports to add even awesomer features to Underscore.js! For example:

Keypaths to reference deeply into nested objects:

_.keypath({hello: {world: ‘!’} }, ‘hello.world’); // returns ‘!’

Array or object _.remove:

var removed = _.remove({bob: 1, fred: 3, george: 5}, [‘bob’, ‘george’]); _.isEqual(removed, [1,5]);

And some other stuff, including stuff that's already been moved into separate independent libraries!

Underscore.string

Underscore.string adds various string manipulation operations like you'd find in Python like capitalization, whitespace removal, searching, and lots more. Actually, Underscore.js isn't a dependency but with _ you get to do chaining.

Link: https://github.com/epeli/underscore.string

Singool.js

Lightweight JavaScript framework for developing single-page web applications.

Singool already uses Underscore.js since that is a dependency for using Backbone.js. So it uses its templating system by default.

Link: http://fahad19.github.com/singool/

Happy.js

"Lightweight, extensible form validation plugin for jQuery/Zepto.js, HTML5, and Underscore.js" - Well, honestly the only Underscore.js integration is has is to allow you to call _.mixin on your own validation functions.

http://thechangelog.com/post/3028809528/happy-js-form-ishappy-lightweight-extensible-form-valida

Underscore alternatives

http://lodash.com/ - An alternative to Underscore.js, delivering consistency, customization, performance, and extra features.
http://sugarjs.com/ - Sugar is a Javascript library that extends native objects with helpful methods. It is designed to be intuitive, unobtrusive, and let you do more with less code. "Underscore with Prototype-ish syntax."

With Views now in Drupal core as of D8 the two most common basic contrib modules from previous versions of Drupal, CCK and Views, are now both part of core.

CCK, now called Entities, from one perspective is what allows us to create arbitrary database tables through a web interface where we can hold arbitrary data yet work with it through a common interface, and use Drupal goodies like widgets, theming, content creation forms and pages, etc. on them. Views is the complement to this way of creating custom content. Views in core means we can now retrieve or query that arbitrary, custom content in arbitrary and custom ways.

Views is basically a web UI for an SQL query builder. The fields are the SELECTed database columns and the filters are your WHERE clause.

This isn't just a feature for non-technical dudes and non-developers. Just because you can write SQL queries doesn't mean you should write all of them. Views gives you an interface into various Drupal core and contributed modules and the content they manage, if those modules integrate with Views via hooks. Entities (or CCK) can create a mess of database tables. Sometimes these tables will change structure from under your feet due to some changes via the admin interface. You might not know until your code is broken. But using Views, the query will always be up to date.

Now that Views is in core, other parts of core could possibly depend on and build on Views, just like many other parts of Drupal core use concepts that came from Entities (CCK). So Drupal core can do a lot more out of the box. And contributed modules can be written to depend on only Entity and Views, which are part of core, and so these new modules won't have any other 3rd party dependencies. That's a win for module developers.

On a Drupal 6 site where I had copied over a database to be used with a freshly checked out tree, I suddenly got this fatal error which stopped any page from loading:

Fatal error: Call to undefined function: block_list() in .../theme.inc on line 935

I'm not sure how it happens. And even after it happens, it's not always immediately a problem. It can be a problem that manifests itself when migrating or copying a database from one site to another. And for some reason, the exact same database worked on another site.

In my case, the problem was the filenames for some core modules and themes and you can see if that's your problem by running this query:

SELECT count(*) FROM system WHERE filename LIKE '%modules/modules%' OR filename LIKE '%themes/themes%';

If that returns anything (like a filename like 'modules/modules/block/block.module') then you have a problem which you can fix by running:

UPDATE system SET filename = replace(filename, 'themes/themes', 'themes');
UPDATE system SET filename = replace(filename, 'modules/modules', 'modules');

On Saturday December 8th, Ho Chi Minh City along with 200 other cities around the world celebrated a global Code Retreat.

What is a code retreat?

Like a meditation retreat, a code retreat is a time set aside to step out of the daily life and routine in order to get deep into practice. It's a full work day, and everyone attending is giving up their Saturday to do what they are probably paid to do from Monday to Friday. That's dedication to the craft. Not so many people came, partly due to poor advertising.

The code retreat was open to anyone but all attendees were fellow programmers, those with some experience already but who want to improve themselves. Besides myself, there was a motley crew of foreigners - an organizer from Japan, a local software tycoon originally from Spain, a Czech developer. Otherwise, all Vietnamese, including startup developers, university faculty, students, and corporate workers.

The venue was ERC Vietnam, the Singaporean MBA school who, along with two other Singaporean schools in Vietnam, is rumored to have lost its operating license from the Ministry of Education which would force it to close. However, the school was still open as usual. A Toastmasters club was also meeting in the room across the hall. There were scheduled outings for students posted in the elevators to be held at local bars and restaurants.

The coding

There were six sessions lasting about an hour each throughout the day and each session was the same both in format and content. The format was:

15 minutes - pair up with another programmer and discuss the strategy for writing the program - choose a language, maybe a framework, discuss data structures and algorithms
15 minutes - one person "drives" the keyboard, the other "navigates" - pair programming (an extreme programming methodology) where two developers use a single laptop to collaboratively develop software
15 minutes - switch up driver and navigator, although often times only a single person was familiar with the text editor or developing environment or even language and API (some languages heavily depend on libraries and APIs to do simple tasks which are difficult to do in the core language alone)
15 minutes - retrospective and break - we wrote down our learnings on sticky notes and collected them for each other to read and at the end of the day we had a whole day retrospective

What were we writing?

We were implementing Conway's Game of Life. Each pair (each session we picked new partners) wrote the game six times throughout the day, erasing the code they had written at the end of each session. The goal was not to write the game of life, but to get better at writing code by writing the same code over and over, by developing the same software over and over.

Halfway through the day some rules started to be added. First, we had to use TDD (test driven development) methodology (which some people didn't realize was feasible in JavaScript), then OOP (what if someone had chosen Scheme or some other functional language?). This was to encourage us to try new techniques and learn them from each other.

I was an outlier in that I wanted to use languages like Python, JavaScript, or even PHP instead of .Net framework languages. Instead of a massive IDE I was using Vim. In the end, in each session where it was up to me I chose Javascript. There was no need to build a new project from a template and then compile executables before running them. Just open a text file, save as html, and use Chrome as a development environment.

Later, more rules were imposed:
No if statements.
No loops (for, while, etc.).
No functions longer than a few lines (maybe 3 or 5 statements max).

We were encouraged to pick one rule and try to work within its constraints. I decided to apply all the rules!

These rules would be easy to abide by using a functional programming language but they're not impossible to follow in Python or Javascript either. But to replace loops using recursion, one thing I found was that nobody remembered how to write recursive functions and most people didn't even want to try as it was hurting their heads.

After the day was over, I took a look at underscore.js, a utility library for JavaScript which makes functional-like programming in JavaScript much easier.

Here's what I came up with.

<script src="underscore.js"></script>
<script>
 
// row and col are indices into 2d array 'grid', either 'today' or 'tomorrow'
function value(grid, row, col) { return (_.isUndefined(grid[row]) || _.isUndefined(grid[row][col])) ? 0 : grid[row][col]; }
function count_neighbors(grid, row, col) {
    return value(grid, row-1, col-1) + value(grid, row-1, col) + value(grid, row-1, col+1) +
           value(grid, row  , col-1) +                           value(grid, row  , col+1) +
           value(grid, row+1, col-1) + value(grid, row+1, col) + value(grid, row+1, col+1);
}
function next_state_if(neighbors, state) { return (neighbors == 3) ? 1 : ((neighbors == 2) ? state : 0); }
function next_state(grid, row, col) { return next_state_if(count_neighbors(grid, row, col), grid[row][col]); }
function calculate_tomorrow(today, tomorrow) {
    _.each(today, function (foo, row) {
        _.each(this.today[row], function (foo, col) { this.tomorrow[row][col] = next_state(this.today, row, col); }, this);
    }, {today: today, tomorrow: tomorrow});
}
 
// helpers
function copy2d(ary2d) { return _.map(ary2d, function (list) { return list.slice(); }); }
function print_r(ary2d) { console.log(_.map(ary2d, function (list) { return list.join(''); }).join('\n')); }
function random_init() { return _.map(_.range(30), function (list) { return _.map(_.range(30), function (list) { return _.random(1); }); }); }
 
// demo main loop
var today = random_init();
var tomorrow = random_init();
print_r(today);
setInterval(function(){
    calculate_tomorrow(today, tomorrow);
    print_r(tomorrow);
    today = copy2d(tomorrow);
}, 400);
</script>

Before Drupal 8: Background

In older versions of Drupal, a lot of "developing" a Drupal site was more clicking around in admin pages rather than writing PHP code in files. This was also Drupal's strength in that non-developers could and developers alike could create powerful and expressive features through a web interface. The downside of this is that changes weren't tracked and there was no way to revert changes - something common and easy to do with any code in a source control system.

Drupalists came up with various solutions. For some major contributed modules like Views and CCK it was possible to export and import views and content types. It was also later possible to package up some changes in some more modules into Features using the Features module and API. But many modules didn't use any API and just haphazardly wrote their configurations to various database tables. To track these and be able to push changes out to other Drupal installations one had to write code in hook_update steps to upgrade the database. This was prone to error as it was being written manually.

And so most Drupal sites were staged and then pushed out to production via database dumps and restorations. Unfortunately, there's no clear separation between configuration and content in a Drupal database. And there's no common way to export, push, and import nodes or any other content from a development site to a production site or vice versa.

With Drupal 8 there is now a solution. Now all configuration that goes through the core configuration API will be written to the database but also to regularly named configuration files in YAML format (not XML as some documentation still mentions). So you can now put the directory containing your configuration into git or svn or whatever. Then you can use your version control to revert configuration changes! And pushing Drupal's configuration changes is simply a matter of committing the automatically generated config files and then updating on the receiving end. Once updated config files are in the right directory they can be synced via the admin interface which shows you a list of changed config files.

How does the new Drupal 8 configuration API work?

Some pertinent background files:

core/includes/config.inc: This is the API for configuration storage.

sites/default/default.settings.php: (from the comments)
+ * By default, Drupal configuration files are stored in a randomly named
+ * directory under the default public files path. On install the
+ * named directory is created in the default files directory. For enhanced
+ * security, you may set this variable to a location outside your docroot.

The following is the directory that was automatically created by installation but you can (and probably should) move it to another location, above the web root, and set it in settings.php.

sites/default/files/config_dd-NwbWruIxwygcszFOmRMZgufouo9QDGaG9UufzfxU/staging: (This is in the secret, private, randomly-named directory created by installation whose name is stored in your settings file which should not be readable by anyone else. The directory name there will not be the same on your own installation.)

This directory contains configuration to be imported into your Drupal site. To make this configuration active, see admin/config/development/sync. For information about deploying configuration between servers, see http://drupal.org/documentation/administer/config

sites/default/files/config_dd-NwbWruIxwygcszFOmRMZgufouo9QDGaG9UufzfxU/active/book.settings.yml (same as the file in core/modules/book/config):

Here's what the YAML looks like:

allowed_types:
  - book
block:
  navigation:
    mode: 'all pages'
child_type: book

And then this file:

sites/default/files/php/service_container/cca23ded514a7eee056b17f0533030d74889c56f42a34ff3ac35597553e2726cf.php
                    ^^^ unreadable          ^^ random, with randomly named PHP class

As noted, "sites/default/files/php/" is not readable.

Instructions on using the configuration API

On your development server, perform whatever configuration changes are needed through the UI. Create Views, check checkboxes, etc. These changes will get written to *both* the database table *and* the file system so the two are in sync (it will also re-generate the digital signatures of the underlying files).

When finished with your edits, review the changes in the config directory with $vcs diff (or your tool of choice) to confirm they look as expected.

If everything looks good, move the changed files to your production server in the usual way (SFTP, $vcs add/commit/push, $vcs update/pull). Nothing will immediately change, as your site will still be reading from the active store.

Finally, go to admin/configuration/system/config or run drush config-update (note: final location/command likely different; making these up right now). This will outline the files that have changed. If the changes seem good, go ahead and confirm to overwrite the content of the active store with the on-disk configuration. The admin tool will offer to regenerate the file signatures if necessary.

So you use the web interface to make configuration changes as you always did. But now those changes will also be written out to disk where you can keep track of changes using git or subversion or whatever VCS you use. Every Drupal 8 module should load and save configuration through the new API and so we get these YAML-format human-readable configuration files for free.

Viettel, Vietnam's military-owned telecom, has opened up direct Facebook integration via sms to Viettel mobile phone subscribers. This is an officially supported way for Vietnamese people to access Facebook on the go without a computer or smartphone. This is targeting Vietnamese people who mainly access the internet at internet cafes and so using this they won't need to hang around at cafes where they're charged hourly while they wait for someone to comment or like their status updates. This applies to a lot of Vietnamese folks and could lead to even faster adoption of Vietnam's #1 social network. What does that imply about any official government ban on Facebook?

I develop on my MacBook Pro and when building Drupal 6 sites, I had to run PHP 5.2 because there were many conflicts in Drupal 6 core and contributed modules which meant running on PHP 5.3 either threw up errors or didn't run properly. Since I run XAMPP for Mac OS X, I also run phpswitch which lets me switch from XAMPP using PHP 5.2 to PHP 5.3 and back.

Unfortunately, Drupal 8 not only requires PHP 5.3 but PHP 5.3.5 or higher. Drupal 7 ran fine on PHP 5.2.5 or higher with PHP 5.3 recommended.

1) XAMPP Mac OS X 1.7.3 comes with: Apache 2.2.14, MySQL 5.1.44, PHP 5.3.1.

From the October 31st to November 11th I attended a Vipassana meditation course/retreat on the outskirts of Ho Chi Minh City.

On the fourth day, we got to eat: yogurt. I was so happy that day.

Five Precepts: No killing, no stealing, no sexual misconduct, no lying, no drinking alcohol

In the past, when looking to hire people to work for your company, you would rely on your personal network and you still have to do that to a great extent today. But if you're trying to fill hundreds of positions then you can't only take personal recommendations.

Before the Internet, there were newspapers which made money by selling subscriptions and selling newspapers at newspaper stands, selling advertising, and selling space in their classified ads. It is still possible to put up classified ads to get employees for your restaurant, bar, massage parlor, hair salon, etc. Print classified businesses like Mua&Ban (Buy&Sell) have also gone online.

Nowadays, instead of Dice or Monster.com or other job websites that are popular in the US, in Vietnam we have three major employment websites: Vietnamworks.com, Kiemviec.com, Timviecnhanh.com. Vietnamworks (Navigos) is the premier and most costly of them all, but the other two get a fair number of CVs posted if you're looking.

Hiring Process

Once you've placed your ad and received some resumes/CVs or searched and found some by yourself, you'll start calling them up and scheduling them to come meet you for an interview. You can have them come to your office or meet them in a cafe. If the candidate isn't so fluent in English, then bring a Vietnamese person to help communicate.

Some problems you may face

They won't show up to the interview or they will show up late.

They will list skills they don't have.

Even if the candidate says they are fluent in English they may not really be comfortable speaking English with you at an interview. Unfortunately this means they don't have great English and if you can only communicate in English with them then this could be a problem unless you plan to only communicate via chat or email. Having a Vietnamese person on hand really cuts down on confusion. Being able to speak English well can easily double a person's salary.

Interviewing software developers

Since I'm usually cost conducting interviews with programmers I like to sit in front of a laptop with the interviewee and have him write some code while I watch. It should be in in language that he is comfortable with and if he's a good programmer he should be comfortable and at least 1 language.

Otherwise the programmer will spend a lot of time having to lookup function definitions online. I also like to use interactive programming environments and so using javascript inside Chrome developer tools is readily available and hopefully familiar to the programmer if not Firefox's Firebug.

Tips

Most cafes are not ideal for conducting an interview because they're too noisy for too loud or to smokey or otherwise just not very professional. A cafe that suits you for drinking coffee or working alone may not be right for meeting others professionally. But try to find a few cafes which have some rooms sectioned off, are non-smoking, and don't have music or movies playing.