As one of the top Drupal firms in the market, we get a lot of questions around Drupal 8 and its broad range of functionality, including Drupal 8 Batch Processing. We thought to start out the new year, we would offer our primer on Drupal 8 Batch Processing.
What is a batch job?
A batch job or batch processing is the execution of a series of jobs in a program on a computer without manual intervention (non-interactive). Strictly speaking, it is a processing mode: the execution of a series of programs each on a set or “batch” of inputs, rather than a single input.
In English, this means that it allows a computer program to break up a series of tasks into smaller chunks or pieces that run without any manual intervention to trigger.
When would I want to use this?
Drupal 8 Batch Processing jobs are valuable to use when there could be large amounts of data or long processes running that utilize a significant amount of memory. An example would be regenerating all URL aliases on your website. The “pathauto” module sets up a batch process when doing this to regenerate 25 aliases at a time, instead of trying to regenerate an entire site (think 5,000 – 500,000 entities) at one time that might cripple the system.
Why would I want to use this?
Performance & Scalability are the biggest reason to utilize Drupal 8 Batch Processing in your development. Batch jobs allow the processing of large amounts of data without relying on a single process to complete the task from start to finish in a single execution. This allows your server resources to be utilized in smaller chunks and freed up after each batch execution finishes.
Here are some questions you can ask when determining if you might need to create a batch process:
- Does the action I need to perform against these items have a per-item resource cost?
- If the action your performing requires loading or processing of each item individually, you should be looking to use batch processing to handle it. If you are performing a simple task, such as a bulk DB query that impacts all nodes in your database, it may not be required.
- Do I need to perform an action on a large number of entities?
- If the answer is yes, than you will likely gain significant performance benefits by utilizing batch processing to work through your task.
- Is there a finite set of data that I am performing actions on or can the dataset grow?
- If you are unsure about how big your data set will get, you should strongly consider batch processing. Not planning for this upfront could cause site downtime and lots of headaches later down the road.
- Even if your current data set is small, can it expand?
- For example, maybe your site only has 30 nodes at the moment, but that number will increase in the future. If this is the case, or you are building a module that you may want to contribute back to the community, you will likely want to look at batch processing as an option for handling this action.
How do I do this?
Creating a batch process in Drupal 8 is relatively straightforward. Here is what you will need to get started:
Demo_batch
- src/Controller/DemoBatchController.php
- demo_batch.info.yml
- demo_batch.routing.yml
- demo_batch.mybatch.inc
Demo_batch.routing.yml
The routing file defines a route, the Controller to be used, and the requirements to use it.
DemoBatchController.php
The controller tells Drupal what to do when the route (defined above) is accessed. In this case, we are creating a Batch Controller which will handle the processing of the batch job.
demo_batch.mybatch.inc
This includes file provides the callback functions for the controller to handle execution of the job. In this example, we are running through a migration task.
Looking for help with your Drupal 8 development? Contact Us to find out how we can help.
Are you still on Drupal 6?
Whether you operate a blog, a small business page, or a large corporate website, if you are still operating on Drupal 6 it is past time for you to consider upgrading your operations. Drupal 6 officially reached End-of-Life on February 24th, 2016. Since then, the Drupal community has largely abandoned support for this version of Drupal.
Why does this impact me?
Anyone running their business on a Drupal 6 version should be concerned because the Drupal community has moved on. As an open source platform, the vitality of Drupal depends on contributors adding new components, modules, features and more. When the community moves on to new versions of Drupal, the older versions are no longer receiving the care and attention required to maintain modern content platforms.
By not adopting Drupal 8, you no longer have the community actively working on modules, bugfixes, improvements, and most importantly, security updates and improvements. The Security team is no longer providing Advisory updates for Drupal 6, and the majority of modules are no longer supported by the maintainers. The bottom line? If you are still operating on an old version of Drupal, you are exposing your site to unnecessary security risk.
Uh oh…so how do I make the switch to Drupal 8?
It’s easy! Upgrading your Drupal site using a migration approach. The first step is to decide how you want to handle your upgrade. Use these questions to help you get started:
- Am I happy with how my website functions and how it looks?
- Does my website/brand need a refresh or update?
- Do I want to refresh my content as part of this upgrade?
- What are the key functions that my website needs to perform?
Based on your answers to the questions above, you will likely fall into one of two categories: Lift & Shift or Website Rebuild.
Lift & Shift: As is migrations
Drupal 8 has made significant strides to make the upgrade process from Drupal 6 to Drupal 8 as smooth as possible. Mike Ryan and others have made a huge impact in this space, rewriting the migrate module from the ground up and including it in the Drupal 8 core. The fact that this was added as a Drupal core initiative speaks volumes of the importance that has been placed on this by the community.
If your site is mainly brochureware, the upgrade process from Drupal 6 to Drupal 8 should be quick and relatively painless. The new migrate module not only migrates content, but it will also migrate data structures through a configuration migration tool. Wait, what? I can migrate my content types and taxonomy terms so that I don’t need to rebuild them!? The answer is YES! (…with some caveats of course).
For more information about the Drupal 8 migrate initiatives and upgrade documentation, check out the “Upgrading from Drupal 6 or 7 to Drupal 8” section on Drupal.org.
Website Refresh: Rebuild Content, UI and UX
So, I don’t have a basic brochureware site but I really need to refresh my brand. Can I still use these tools to take some of the load off of my shoulders? The answer is yes, but in a more limited fashion.
Depending on the size of your website, utilizing the D6 to D8 migration tools may still save you time and money. This will ultimately depend on how significant the changes to your information architecture and content will be.
If you have a large number of pages (let’s say >1000) and some percentage of these pages will remain relatively in-tact from an information architecture perspective (Blogs, Resources, Press), then you will likely still benefit from investing time in migration scripts.
If you are planning to refresh/rewrite all of your content AND provide a new UI/UX on your website, the effort to create a migration script may be higher than handling the process manually.
So, how do I get started?
If you are planning to move forward with an upgrade, it will be useful to check out the following links before you start:
- Upgrading from Drupal 6 or 7 to Drupal 8
- Upgrade using Drush
- Known Issues when migrating from Drupal 6 or 7 to Drupal 8
While the idea of being able to be generic enough to meet a wide variety of peoples needs, it is unlikely that your migration to Drupal 8 will be handled out-of-the-box. In the likely case that you will need to customize, here are some more useful links:
- Migrate Process Overview (Processor Plugins)
- Webinar – Migrating to Drupal 8
- Carlyle Example Migrate Project
Would you like help creating a more detailed plan for migrating your website to Drupal 8? Contact Us – We would be happy to help!
What is Render Caching?
Have you ever found yourself asking the question “Why are my changes to my twig template not showing up?” or “Why do I have to clear my cache every time I make a change to a preprocess function?”. The answer is: Render Caching. Render caching is the process of storing post-rendered data so that Drupal does not needlessly rebuild and re-render arrays.
Why Render Cache?
The process of loading, rebuilding and re-rendering entities in Drupal is a very expensive task and much of the time, is unnecessary. Render caching allows us to bypass much of the unnecessary processing and dramatically improve the site performance.
So… What does the Render Cache look like?
The first step in your journey is to understand what makes up the Render Cache property:
- Cache Keys – a representation of a set of code that you want to make cacheable, and is typically something that is too expensive to render on every page load.
- Cache Contexts – variations of what I am rendering. Does my display change based on user role, language, time of day, location, etc?
- Cache Tags – custom identifiers that this render array depends on to render properly, such as Referenced Entity ID’s
- Cache max-age – determines how long the item I am rendering can be cached for. This should always be set.
The render cache is a “#cache” property that is defined on your render arrays. You are able to add and/or customize this for any render element.
Plan for caching
Working with render caching is something that you should be thinking about from the beginning of your project.
Some questions that may help you in the early stages of your project would be:
- How often are pages on my site updated?
- Are there any highly dynamic sections of the website? (ie. Stock Ticker, Countdown Timer, etc…)
- Is there any content that is context specific? (ie. Location based, user role based, etc…)
- Is the website complex enough to require custom render processing?
The importance of cache tags
Cache tags are where the magic happens! Cache tags allow the system to identify dependencies to a render object and invalidate pages when there are changes made to the dependent page.
As mentioned above, cache tags are custom identifiers that this render array depends on to render properly, such as Referenced Entity ID’s.
Real World Scenario:
Lets walk through an example: We have a website that has a blog post node/123: “My Drupal 8 Render Test” which is referenced in the recent content section on the home page, featured on a few of our product landing pages and in the related content of other blog pages.
In the example above, assuming that we have followed best practices and rendered our cache property arrays, the cache tag of “node:123” should have been added to the render tags array of the homepage, product landing pages and relevant blog posts that reference this item.
Now, we want to change the title of that page to be “Drupal 8 Render Caching”. When I save my node to make this update, it will trigger a cache invalidation of all nodes that have “node:123” in their cache tag list, thus ensuring that my new title shows up on pages where it is referenced.
For more advanced cache purging (memcache, varnish, CDNs), you will need to utilize the purge module. Acquia has recently launched their Public Beta for Acquia Purge as well, so if you are hosting on Acquia, it simplifies the process significantly.
There are more useful tidbits in the presentation that John Doyle, our CTO, gave at the DC Meetup in October.
Some additional community information around render caching:
- Disable Drupal 8 Render Caching for Development: https://www.drupal.org/node/2598914
- Drupal 8 Render Cache API: https://www.drupal.org/docs/8/api/render-api
- Cachability of Render Arrays: https://www.drupal.org/docs/8/api/render-api/cacheability-of-render-arrays
Having worked with Drupal 8 in a production setting at one of the top development agencies for the last 15 months, I feel that I can responsibly say that Drupal 8 is ready for prime time. In fact, given all of the great improvements that the platform has to offer, it’s hard to think of an scenario where I would recommend Drupal 7 to a client. These include a standardized Symfony2 framework, a twig templating system, partial page caching, configuration management, layouts, and much more.
To learn more about Drupal 8’s new features, I spent a day recently at DrupalCon Baltimore, an experience that has heightened my excitement about Drupal and the future of the platform. Here are the takeaways that we got from the conference:
- Focus on Lowering the Barriers of Entry
- Core initiatives targeted at improving content authoring
- Revamped Release Cycles
- Drupal maturing in large enterprise
Opening the Flood Gates
The Driesnote was amazing as always. This one was more exciting than usual as there was a strong emphasis on the community and the shift for Drupal to become more user-friendly and lowering the barriers of entry. From a technical standpoint, the standardization on the Symfony2 framework and the addition of the twig templating system make working with Drupal more attractive to PHP developers, opening the platform up to a much wider developer market. From the content side, Dries highlighted the work being lead by Keith Jay to provide a better out-of-the-box experience to all users.
Content is King
In an ever changing market, it is important to stay ahead of the curve and adapt your organization to meet the needs of your client base. We validated a big shift that we are seeing in the market where the decision-maker is no longer the IT team – It has shifted to the marketing team. It is great to see Drupal follow this trend with the strong focus on the new core initiatives around UX, such as layouts and in-place editing. Dries also highlighted Cristina Chumillas for her work in improving the UX of several core pages.
Maintenance made easy
Another exciting announcement was around the revamp of the Drupal release cycles to make core upgrades for both minor and major versions of Drupal easier. The new 6-month cycles have been running great, and I for one am excited to see it. In this new model, functionality will slowly be deprecated (instead of removed) throughout the minor release versions as new functionality is added. This will give module developers an extended period of time to upgrade. Major releases will go one step further and remove the set of deprecated functionality to start the codebase off on a clean slate.
Climbing the Corporate Ladder
Drupal continues to gain traction in the large enterprise space with organizations and marketing teams looking to spend more of their budgets on content and campaigns rather than recurring subscription fees. This can be validated by the uptick in features and functionality that the community is providing for Drupal 8. As the market changes, so should the technology. The greatest thing about Drupal and the community around it is that we are the ones choosing the direction of the platform. We have thousands, or even hundreds of thousands of people validating this platform in the market and pushing the direction of the platform forward.