Chromedp Concurrency with Multiple Tabs
![Profile image of Atakan Demircioğlu](/images/atakan.avif)
Understanding how to navigate chromedp concurrency with multiple tabs can significantly enhance your browser automation projects. This article guides you through the process, offering practical tips and clear understanding of this unique tech tool.
What is Chromedp?
For starters, chromedp is a programmer's ally. This powerful tool allows software developers to navigate the web in a programmatic way. Basically, it's a Go package that makes browser automation easier as it allows you to control Chrome (or any other Chrome-based browser) over the DevTools protocol. This allows you to simulate a user's operations on the browser, perform automated tasks, and much more.
func main() {
startTime := time.Now()
urls := []SSUrl{
{URL: "https://keeplearning.dev/", Selector: `div.hero`, Prefix: "k"},
{URL: "https://go.dev/", Selector: `img.Hero-gopherLadder`, Prefix: "go1"},
{URL: "https://go.dev/", Selector: `img.Hero-gopherLadder`, Prefix: "go2"},
}
ctx, cancel := chromedp.NewContext(
context.Background(),
chromedp.WithDebugf(log.Printf),
)
defer cancel()
var wg sync.WaitGroup
const grMax = 5 // Number of goroutines to run concurrently
ch := make(chan int, grMax)
for _, url := range urls {
// Increment the wait group counter for each URL
wg.Add(1)
ch <- 1
// Execute tasks concurrently inside a goroutine
go func(u SSUrl) {
defer func() { wg.Done(); <-ch }()
log.Printf("Capturing screenshot for URL: %s", u.URL)
// same browser, second tab
newCtx, _ := chromedp.NewContext(ctx)
captureScreenshot(newCtx, u)
}(url)
}
// Wait for all goroutines to finish
wg.Wait()
endTime := time.Now()
scriptDuration := endTime.Sub(startTime)
fmt.Println("All screenshots captured in => ", scriptDuration)
}
The critical part is this part;
newCtx, _ := chromedp.NewContext(ctx)
In this part, we are creating new tabs and we are using them with the goroutines.
To limit the number of simultaneously running goroutines we are using buffered channel. You can check this example for better understanding.