JavaScript as a serious programming language?

Unbeknownst to many folks, even those developing websites, JavaScript libraries are hot right now. One reason is that they're being used on both the frontend (browser) as well as the server via Node.js. And so Drupal 8 includes two new JavaScript libraries, beyond the mainstay of jQuery, which are popular because they rewrite how we use JavaScript, much like Drupal changes how we develop PHP software. They are the Backbone.js and Underscore.js JavaScript frameworks. Since they're in core, Drupal itself can use them in its own JavaScript code. However, as far as I can tell nothing in core, besides Backbone, is using Underscore yet.

Backbone.js (http://backbonejs.org/) gives structure to web applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing API over a RESTful JSON interface.

The cool thing about Backbone is it splits the abstract representation from DOM events and HTML rendering in your JavaScript code. This is especially applicable to web apps which constantly get new data via AJAX from a REST server and then update the browser in-page rather than reloading a new URL and rendering the page's html on the server - for example via Drupal and it's normal theme/template system.

If you think of modern web applications like GMail or Trello (which is built on Backbone) they work mostly by not reloading the page but rather with each click of the mouse pulling in data from the server then rendering it somehow in the current page. Backbone is how Rdio, Hulu, and Pitchfork play music and videos continuously across "page loads". This can also be happening in the background independent of user activity to update content or display status messages or maybe show new email.

AJAX vs AHAH

In Drupal, you'll see a lot of references to AHAH. AHAH is basically AJAX except that it's honest about the X, which in AJAX stands for XML, but in Drupal using AHAH it's actual HTML which can be inserted into a page. Or it could be JavaScriptON that's parsed and, using some custom JavaScript logic, turned into HTML by manipulating the DOM.

Without something like Backbone, this is how AJAXy applications are written in Drupal. We add in #ahah hooks that form elements and their widgets can use (when JavaScript is available.) But the login on the frontend is completely custom and usually is jQuery $.ajax callbacks that use jQuery selectors to find and replace DOM elements manually.

Backbone.js gives us a framework to represent things we want to update on the frontend in a structured way, with clear separation between the model and rendering. The model language directly maps to the MVC of Ruby on Rails. Then by updating the model, which is all the logic we really want, the visual representation is updated for us. Backbone can call Underscore templates, Mustache.js, or directly insert into the DOM server-side rendered (the old way we just mentioned) HTML.

So now our AHAH callbacks can return JSON that's pure data, and allow Backbone to handle rendering the HTML.

So what about Underscore.js?

Backbone itself depends on Underscore, but Underscore by itself is also useful. For example, see how Conway's Game of Life can be implemented in JavaScript without if-statements or loops (for, for-in, while) by using Underscore.js and by programming in a more functional style, with functionality similar to Ruby or Python. jQuery, long since included in Drupal core, also offers some functions like $.map or $.each but Underscore is much more complete. Underscore is a JavaScript utility library, not for writing any specific kinds of apps but for making all of your JavaScript code potentially cleaner, simpler, shorter.

Some _ functions:
_.each
map
reduce
find
filter
every
pluck

Underscore also has some array and object helper functions, deep === comparison (isEqual), etc. See it all at http://documentcloud.github.com/underscore/

You can play around with Underscore.js right now by writing up some code at http://jsfiddle.net/

On a Drupal 6 site where I had copied over a database to be used with a freshly checked out tree, I suddenly got this fatal error which stopped any page from loading:

Fatal error: Call to undefined function: block_list() in .../theme.inc on line 935

I'm not sure how it happens. And even after it happens, it's not always immediately a problem. It can be a problem that manifests itself when migrating or copying a database from one site to another. And for some reason, the exact same database worked on another site.

In my case, the problem was the filenames for some core modules and themes and you can see if that's your problem by running this query:

SELECT count(*) FROM system WHERE filename LIKE '%modules/modules%' OR filename LIKE '%themes/themes%';

If that returns anything (like a filename like 'modules/modules/block/block.module') then you have a problem which you can fix by running:

UPDATE system SET filename = replace(filename, 'themes/themes', 'themes');
UPDATE system SET filename = replace(filename, 'modules/modules', 'modules');

Before Drupal 8: Background

In older versions of Drupal, a lot of "developing" a Drupal site was more clicking around in admin pages rather than writing PHP code in files. This was also Drupal's strength in that non-developers could and developers alike could create powerful and expressive features through a web interface. The downside of this is that changes weren't tracked and there was no way to revert changes - something common and easy to do with any code in a source control system.

Drupalists came up with various solutions. For some major contributed modules like Views and CCK it was possible to export and import views and content types. It was also later possible to package up some changes in some more modules into Features using the Features module and API. But many modules didn't use any API and just haphazardly wrote their configurations to various database tables. To track these and be able to push changes out to other Drupal installations one had to write code in hook_update steps to upgrade the database. This was prone to error as it was being written manually.

And so most Drupal sites were staged and then pushed out to production via database dumps and restorations. Unfortunately, there's no clear separation between configuration and content in a Drupal database. And there's no common way to export, push, and import nodes or any other content from a development site to a production site or vice versa.

With Drupal 8 there is now a solution. Now all configuration that goes through the core configuration API will be written to the database but also to regularly named configuration files in YAML format (not XML as some documentation still mentions). So you can now put the directory containing your configuration into git or svn or whatever. Then you can use your version control to revert configuration changes! And pushing Drupal's configuration changes is simply a matter of committing the automatically generated config files and then updating on the receiving end. Once updated config files are in the right directory they can be synced via the admin interface which shows you a list of changed config files.

How does the new Drupal 8 configuration API work?

Some pertinent background files:

core/includes/config.inc: This is the API for configuration storage.

sites/default/default.settings.php: (from the comments)
+ * By default, Drupal configuration files are stored in a randomly named
+ * directory under the default public files path. On install the
+ * named directory is created in the default files directory. For enhanced
+ * security, you may set this variable to a location outside your docroot.

The following is the directory that was automatically created by installation but you can (and probably should) move it to another location, above the web root, and set it in settings.php.

sites/default/files/config_dd-NwbWruIxwygcszFOmRMZgufouo9QDGaG9UufzfxU/staging: (This is in the secret, private, randomly-named directory created by installation whose name is stored in your settings file which should not be readable by anyone else. The directory name there will not be the same on your own installation.)

This directory contains configuration to be imported into your Drupal site. To make this configuration active, see admin/config/development/sync. For information about deploying configuration between servers, see http://drupal.org/documentation/administer/config

sites/default/files/config_dd-NwbWruIxwygcszFOmRMZgufouo9QDGaG9UufzfxU/active/book.settings.yml (same as the file in core/modules/book/config):

Here's what the YAML looks like:

allowed_types:
  - book
block:
  navigation:
    mode: 'all pages'
child_type: book

And then this file:

sites/default/files/php/service_container/cca23ded514a7eee056b17f0533030d74889c56f42a34ff3ac35597553e2726cf.php
                    ^^^ unreadable          ^^ random, with randomly named PHP class

As noted, "sites/default/files/php/" is not readable.

Instructions on using the configuration API

On your development server, perform whatever configuration changes are needed through the UI. Create Views, check checkboxes, etc. These changes will get written to *both* the database table *and* the file system so the two are in sync (it will also re-generate the digital signatures of the underlying files).

When finished with your edits, review the changes in the config directory with $vcs diff (or your tool of choice) to confirm they look as expected.

If everything looks good, move the changed files to your production server in the usual way (SFTP, $vcs add/commit/push, $vcs update/pull). Nothing will immediately change, as your site will still be reading from the active store.

Finally, go to admin/configuration/system/config or run drush config-update (note: final location/command likely different; making these up right now). This will outline the files that have changed. If the changes seem good, go ahead and confirm to overwrite the content of the active store with the on-disk configuration. The admin tool will offer to regenerate the file signatures if necessary.

So you use the web interface to make configuration changes as you always did. But now those changes will also be written out to disk where you can keep track of changes using git or subversion or whatever VCS you use. Every Drupal 8 module should load and save configuration through the new API and so we get these YAML-format human-readable configuration files for free.

Say you have a bunch of blocks and you want them to be displayed on certain nodes of varying content type based on some criteria like the content type and some CCK fields or taxonomy. You can't do this with the stock block visibility settings without writing custom PHP code.

But we can implement it using some existing basic Drupal building blocks: CCK and Views

1. Create a content type called Visibility Block.

You might have a field for content type where the possible values are returned from code which returns an array of the content types (using function node_get_types()).

2. Then for any fields you want to match, you'll have the same fields in this content type. For example, if one of your content types has a textfield and the possible values are 1, 2, 3, then do the same for Visibility Block.

When you create a Visibility Block, you'll have your block content in the body as normal (optionally you could use Block Reference and create blocks like usual and then link to them in the node instead, but I see no point in the extra effort and redirection), then select the conditions for the pseudo-block being visible.

3. Now create a view called Visibility Blocks Viewed. You'll create block displays, one for each content type that you have a Visibility Block set for which may only be one or two of your c-types.

Create an overridden argument each block display for the content type -field- in Visibility Block. You want to match the content type of the viewed node with the field in the Visibility Block, which are not the same type of thing. So you'll need to convert the argument in code.

You'll call menu_get_object() to get the $node because it's better than "$node = node_load(arg(1));". For the chosen c-type for that block display you will check that the implied node's c-type is what ever type you want to show in this block display because you will also check any fields that are specific to this c-type. Use PHP to supply a value since no argument will be passed in, and have the code load the current node and return the type. Then use PHP for the Validator Options and if the c-type doesn't match then you will display empty text. If you have multiple c-types which share CCK fields then you can put them into a single block display.

4. For each content type, in the chosen block display you will create a new argument for each compared against CCK field. You will pick a field from V-block and then use PHP to return a default value of the currently viewed node's field's value. You won't need to do Validation on these arguments.

5. Finally, you may want to limit the number of nodes returned, maybe just one. Now give this block a title and save it and configure its region in the normal block admin.

--

Why not the normal block visibility settings?

Because there you can't even configure by content type (anymore in D6). You can configure by path glob and by PHP code which overrides the path glob field (including in the database).

So under "Page specific visibility settings", set to "Show if the following PHP code returns TRUE (PHP-mode, experts only).", the field is blocks.pages. Essentially, PHP code works by overriding the pages list.

Why not use Block Page Visibility?

Block Page Visibility (http://drupal.org/project/bpv) enables site developers to centralize the display of blocks to a single PHP function. It is an alternative to controlling display via each block's configure form. The more "sometimes on, sometimes off blocks" that a site uses, the more useful this module becomes.

This takes over the visibility settings of all your blocks by calling:

$sql = 'UPDATE blocks SET visibility = 2, pages = CONCAT("<", "?", "php ", "return bpv_is_visible(\'", module, "-", delta, "\'); ", "?", ">") WHERE status=1 AND theme=\'%s\'';

Sometimes it's easier to configure a block by just whether you're logged in, or whether you're on the front page.

This module doesn't give you any finer grained controls. And you have to configure every block in code. You also lose all your current block visibility settings once you install this module. To use this module you have to implement your own bpv_config or bpv_configuration (I think it's a bug that it's looking for bpv_configuration but actually uses bpv_config).