Background processing for performance in Rails

The problem

Overloaded systems with poorly optimized processes degrade user experience, impacting both operational efficiency and customer satisfaction.

Performance issues often stem from tasks ill-suited for real-time processing during normal web operations.

Solution: Move Processing into Background Jobs

Long operations within the request/response cycle degrade performance.

Common examples include generating reports, mass data updates, cache refreshes, external API calls, and sending emails.

cron Tasks

For repetitive, batch-like operations, cron jobs are a fitting solution. This method suits tasks with predictable, consistent workloads.

Example: Caching database counts

# File: lib/tasks/cache_counts.rake
task :cache_counts => :environment do
  ItemCountCache.refresh
end

# File: app/models/item_count_cache.rb
class ItemCountCache < ApplicationRecord
  def self.refresh
    update(count: Item.count)
  end
end

While simple, cron jobs may lack flexibility for more dynamic or irregular tasks, where a queuing system might be more appropriate.

Queuing

Queues excel in managing asynchronous tasks that vary in frequency, size, or complexity.

They allow enqueuing tasks as needed, enhancing responsiveness and scalability.

Choosing a Queue System:

For Rails, delayed_job and Resque are robust options.

Resque uses Redis, making it suitable for heavy background work, while delayed_job, leveraging a SQL backend, is excellent for conventional Rails stacks.

But favorite is just sidekiq or I tend to use solid_queue.

Implementing a Job:

# File: app/jobs/sales_report_job.rb
class SalesReportJob < ApplicationJob
  queue_as :default

  def perform(user)
    report = generate_report
    Mailer.sales_report(user, report).deliver_now
  end

  private

  def generate_report
    CSV.generate do |csv|
      csv << CSV_HEADERS
      Sales.find_each { |sale| csv << sale.to_a }
    end
  end
end

# File: app/controllers/reports_controller.rb
class ReportsController < ApplicationController
  def create
    SalesReportJob.perform_later(current_user)
    render plain: "Report generation in progress."
  end
end

This approach adheres to the Single Responsibility Principle and decouples the heavy lifting from web processing, enhancing testability and maintainability.

Keep It Real

While background processing is crucial for maintaining performance, indiscriminate use can lead to an over-engineered and fragile system.

Evaluate whether each task genuinely benefits from being run in the background.

Adding async jobs adds potentially:

• Another DB (e.g: redis if using sidekiq)
• Adding load to the current DB (e.g: if using solid_queue)
• More states to handle (e.g: [:loading, :started, :done, :failed])
• Threading problems

Databases

Despite Rails’ ORM abstracting much of the database interaction, performance tuning at the database level is critical.

This includes proper indexing, query optimization, and data model adjustments specific to Rails applications.

Read more here

Example: Indexing for Performance

# File: db/migrate/20230101123045_add_indexes_to_sales.rb
class AddIndexesToSales < ActiveRecord::Migration[6.1]
  def change
    add_index :sales, :created_at
    add_index :sales, :user_id
  end
end

Indexes improve retrieval times but should be used judiciously to avoid unnecessary overhead on write operations.

Background jobs are an effective strategy for mitigating performance bottlenecks in real-time web requests, achieving a smoother user experience.

By judiciously determining what to run asynchronously, you can maintain a responsive, efficient Rails application.

Related posts