Harnessing the Power of errgroup in Go ⚙️🛠️

power of errgroup in go

In Go, concurrent programming is a core strength, but managing multiple goroutines can be challenging, especially when you need to handle errors effectively. This is where the errgroup package shines, providing a clean and efficient way to manage a group of goroutines with error handling.

To illustrate the features of errGroup, we’ll apply them to a task management app. We’ll start with a basic implementation and progressively improve our code by leveraging the package’s capabilities.

Task Management System

we will build a simple task distribution system for illustrations. We will define a Task interface to do some work.

type Task interface {
 Do(ctx context.Context, workId string) error
}

The Do method will do the required work, returning an error if the work can’t be done.

next, we define a type Tasks that maps a workId to a specific Task implementation as follows;

type Tasks map[string]Task

Using a WaitGroup, we can implement a function that do each work in Tasks concurrently:

func Work(ctx context.Context, tasks Tasks) {

 wg := sync.WaitGroup{}

 // worksemo is a counting semaphore used to
 // enforce a limit of 20 active goroutines.
 worksemo := make(chan struct{}, 20)

 for i, t := range tasks {
  wg.Add(1)
  go func(workId string, task Task) {
   defer wg.Done()
   worksemo <- struct{}{}
   task.Do(ctx, workId)
   <-worksemo
  }(i, t)
 }

 wg.Wait()

}

Can you spot the issue in this implementation?

The issue with our implementation is that we’re ignoring any errors returned from Do. A better implementation would propagate the first error returned from any of the Do calls to Work’s caller.

What is errgroup?

The errgroup package, part of the Go x packages, simplifies working with multiple goroutines by allowing you to wait for all of them to complete while capturing any errors that occur. It provides a structured way to manage goroutines, ensuring that if any goroutine fails, the error is propagated back to the caller.

How to Use errgroup

working with errgroup is similar to how you work with waitgroup

1. Create the errgroup

 eg := errgroup.Group{}
// or 
 eg , egCtx := errgroup.WithContext(ctx)

2. Go(f func() error)

The Go method takes the function f and runs it in a separate goroutine. Internally, Go does the equivalent of WaitGroup.Add(1) before starting the goroutine and WaitGroup.Done() once the function has finished.

Go calls the given function in a new goroutine. It blocks until the new goroutine can be added without the number of active goroutines in the group exceeding the configured limit.

The first call to return a non-nil error cancels the group’s context, if the group was created by calling WithContext. The error will be returned by Wait.

3. Wait() error

Wait blocks until all function calls from the Go method have returned, then returns the first non-nil error (if any) from them.

4. SetLimit(n int) (optional)

SetLimit limits the number of active goroutines in this group to at most n. A negative value indicates no limit.

Any subsequent call to the Go method will block until it can add an active goroutine without exceeding the configured limit.

The limit must not be modified while any goroutines in the group are active.

with our knowledge of errgroup, let’s fix the issue with our task management implementation using errgroup

Task management & ErrGroup

fixing the issue on the Work function, the new implementation should look like this

func Work(ctx context.Context, tasks Tasks) error {

 eg := errgroup.Group{}
 eg.SetLimit(20)

 for i, t := range tasks {
  func(workId string, task Task) {
   eg.Go(func() error { return task.Do(ctx, workId) })
  }(i, t)
 }

 return eg.Wait()
}

Why Use errgroup?

1. Error Propagation: If any goroutine returns an error, it is captured and returned by errgroup.Wait(), ensuring you don’t miss critical failures.
2. Context Integration: errgroup.WithContext() provides a context that can be used to cancel all goroutines if one fails, improving the robustness of your application.
3. Simplified Code: By using errgroup, you can reduce boilerplate code for managing goroutines and focus on the logic that matters.

Summary

The errgroup package is a powerful tool in Go’s concurrency toolkit. It streamlines error handling and coordination between goroutines, making your concurrent programs easier to write, read, and maintain. If you’re working with multiple goroutines in Go, errgroup should be part of your arsenal.

Follow me on X for more amazing Golang tips.