I have a love-hate relationship with GA4.
There have been periods of the last couple of years when I was more ‘love’ than ‘hate’ and found myself defending GA4 whilst most digital marketers were launching tirades of GA4 abuse, but I cannot deny that I have found myself falling into the ‘hate’ camp more often than not recently.
Whilst I will acknowledge that GA4 is fundamentally very flexible and potentially very powerful, there are too many instances when you feel that it *still* isn’t really ready for the mainstream. I wanted to share my latest frustration as an example of why it can be hard to trust the data that is shown to us.
What is ‘user engagement’ in GA4?
I need to wind back and introduce the concept of ‘user engagement’ in GA4. User engagement was sort of responsible for killing off the very familiar ‘bounce rate’ of Universal Analytics (UA).
Funnily enough, the removal of bounce rate as a metric was one of the UA to GA4 changes that I used to defend the most. I always thought that an obsession with bounce rate was not healthy and there were plenty of instances where a high bounce rate did not actually reflect a problem page. For example, a user may have spent some time reading, and enjoying, a blog post on your site before leaving. In UA speak, that was a ‘bounce’ and that was generally a bad thing. The reality, however, was that the user had a great experience and thoroughly enjoyed your content.
GA4 approached things a different way and bounce rate was replaced by the concept of ‘engaged sessions’, which generally means that one of the following criteria was met during a visit:
- the user spent more than 10 seconds on a page (which would have been the case in the theoretical visit above, where a user spent a long time reading your blog post)
- there is at least one conversion event (this requires some thinking through, but there are lots of events that you could set to recognise a positive user experience, e.g. scrolling to various points on a page)
- at least 2 pages are viewed (this is the closest to the old bounce rate metric)
In essence, an engaged session is the inverse of a bounce. You can therefore manually calculate a bounce rate if you are really desperate to cling on to that metric, although GA4 ‘bounce rates’ will almost always be significantly lower than their UA equivalents, so it is hard to compare apples to apples.
If you are more of a geek, you will want to know exactly how the engaged session is marked. There is an analytics parameter called ‘seg’ which is used to indicate whether a session is ‘engaged’ or not. If the value is “0” (which it will be by default), the session will not be included as an engaged session. The parameter is changed to “1” as and when any of the three conditions above are met.
Crucially, an event called ‘user_engagement’ is fired in GA4 when the seg parameter is changed.
If you are familiar with the GA4 DebugView, you can play around on a site and see the user_engagement firing. The easiest / quickest way to see it is to simply navigate to another page on a site, as that should trigger the event.
What is the problem with user_engagement?
All the above sounds very sensible and logical. If you do have a test run in DebugView, you are likely to see the user_engagement event firing more than once.
Staggeringly, leaving the site is an action that seems to fire the user_engagement event a lot of the time (although not all the time, just to be even more annoying). This is an enigma as, in my little brain, leaving a site is probably about as far from engaging with it as I can imagine. Why on earth would leaving be represented by a user engagement event?
This excessive firing of the user_engagement event is very obvious if you look at events in GA4 reports, which will typically show that there are far more user_engagement events than there are individual users generating the events:
As shown above, this particular report indicates that there were, on average, 2.53 engaged_user events per user, confirming that the event is fired more than once during a session.
This is surely bonkers? To me, it is a fairly binary metric – either you have engaged with a site or you haven’t. I fail to see any logic in reporting an event count of any more than 1 per user for the ‘user_engagement’ metric as the seg parameter should only change once, on the first time that one of the three criteria is met.
The issue here is that it really undermines the metric as a valuable indicator of quality. You can, of course, just use the user count as a more accurate indicator of how many people engaged with your content but it does not exactly help build confidence in the platform, especially when you can see the act of leaving a site as a user_engagement event.
Can you use user_engagement events as conversions?
I should really talk about ‘key events’ rather than ‘conversions’, as that is what they are now called, but this is simply a semantic issue – a key event is what we used to know as a conversion…
Although a softer conversion, some users will want to look at the quality of traffic from different sources and could use the user_engagement as an event to mark a good visit. It would normally be a simple case of heading over to the Events tab in the GA4 admin section and marking the user_engagement event as a key event. There is a mini slider type button next to all events, so it is very easy to upgrade an event to become a key event. This will also feed back to other platforms, such as Google Ads, so that you can easily report on conversions key events.
In this particular case, you will run into a problem as you will not see the user_engagement event in the list of events, so you are unable to mark it as a key event. There is an easy workaround though, as you can create a new event that is based on an existing event. For example, this will create a new event called ‘engaged_visitor_session’ which will fire whenever the user_engagement event is triggered:
You can name the new event whatever you choose, just remember what you name it so that you can find it in the reports. You will also need to find it in the events list in GA4 admin and mark it as a key event.
Are you and GA4 friends again?
Whilst setting up the new event is fairly straightforward, I am afraid that it does not do exactly what you would expect it to do. Grrrrrr. No, I am not having a good GA4 day!
I hope you would agree that it would be safe to assume that the total count for the new ‘engaged_visitor_session’ event would be the same as that for ‘user_engagement’ as the new event is triggered by the standard user_engagement event. Surely, the numbers would be the same – the workaround is simply to allow the event to be marked as a key event?
True to GA4’s infuriating form, you will find that the numbers do not actually match up if you come back a few days later:
There is no logical reason why the event count should not be the same and it is infuriating to see such a massive difference. In my humble opinion, it simply calls into question the overall trust that you can have in GA4 data.
There is an argument that the new event is actually a better reflection of the true number of engaged sessions, as the event count per user has dropped from 2.53 (user_engagement) to 1.33. To be blunt, this still really annoys me as there is no explanation as to why the numbers are different at all and it should still only be a maximum of one event per user, as it is a very binary measure! You are either ticking the engaged box or you are not!
GA4 key event counting method
The more GA4 savvy amongst you may well be questioning whether the difference in numbers can be explained by a different counting method being used. That is a good shout, but the data shown above was using the ‘once per event’ setting, so I would humbly suggest that the total event counts should be identical?
For the less GA4 savvy, another advantage of creating the new ‘engaged_visitor_session’ event is that you can specify which counting method to use. This is the case for any key event – you need to click on the three dots at the end of the row in the key events list and then click on ‘Change counting method’.
I have actually now changed the counting method to only count one event per session:
I would expect this to introduce a difference in the total user counts, as we should be eliminating the double / triple counting that is clearly happening with the standard user_engagement event, but I would still expect the user count to be fairly similar. Again, once a user has been designated as an engaged visitor, it is impossible to undo that.
It is early days, but I am still seeing fairly large discrepancies between the two events, despite one being based entirely on the other.
What to do?
To be honest, I am struggling to come up with a solution. Whilst I think that the new ‘engaged_visitor_session’ event will actually reflect a more accurate count of engaged users, as it is clearly addressing some of the overcounting of the standard event (although I do not know why!) even before reducing the counting to once per session, I am not entirely trusting of the data.
More importantly, I am finding it impossible to defend GA4 on this occasion. I have no doubt that a lot of GA4 hatred over the past two years has stemmed from the normal human resistance to change and GA4 isn’t really that bad, but I won’t deny that I have actually been feeling more frustration with it in recent months.
Not only does it take too long for data to actually show up in GA4, I am losing faith in its accuracy. There are simply too many instances where data does not match up (e.g. standard reporting can often show different data to the explore reports) and there is no logical explanation of weird oddities that seem to pop up far too often.
Three years ago, I think it was fair to expect a few teething problems to show up. That should not be the case now, a full year after it was forced upon the digital marketing world. Yes, it is free, but does it really need to feel so cheap?
Come on Google, please do better.
