Introducing Email Meter Enterprise: email statistics and performance management for companies

Email Meter Enterprise

We’re launching today the most advanced real-time email statistics solution for companies big and small. We combine Big Data analysis techniques with our decades of experience in the email statistics field to offer you our new product: Email Meter Enterprise.

Email Meter Enterprise

Email Meter Enterprise is our company email statistics solution for companies of all sizes, no matter how big their email datasets are. Let’s review some of the features!

Email Meter Enterprise – Features

  • Thorough email analysis:  we leverage Big Data tools (such as BigQuery) to monitor your company’s email activity and increase productivity.
  • Powerful dashboards: you get a full set of dashboards: Received, Sent, Google Groups, Response Time, All Mail, More Metrics… That’s roughly 40 (more every week!) different metrics about your email, response time, attachments, etc.
  • Dive deep, filter your data: filter by user, team, date ranges… or mix and match! Get exactly the data that you need.
  • Departments and teams: add users to custom teams and check their metrics with just one click. Comparing users and teams performance hasn’t ever been easier.
  • Searchable list of emails: get a list of all received and sent emails of your domain and easily check if they have been responded or not, and when.
  • Fast and reliable: get access to months of data in a matter of seconds. Analytics are based on the email logs stored by Google on BigQuery, your single source of truth for email data.
Sent Dashboard for Email Meter Enterprise

We discovered that our email productivity is highest between 2pm and 4pm… At 12pm, we’re heads down developing new features & that shows in our email volume. There’s a peak at 8pm, when customers across the globe are contacting us. This data shows that our team is both responsive to customers in different timezones and processing emails to start the next workday at inbox zero.

And this is just the start! We have an exciting roadmap with tons of new features: more filter capabilities, more metrics, email reports, and custom roles so your Data Team can get to the root of performance and efficiency issues 🔥

Email Meter Enterprise is available to companies of all sizes starting today. You just need G Suite Enterprise to get started.

About G Suite Enterprise

Why do you need G Suite Enterprise to use Email Meter Enterprise? Because G Suite Enterprise, the most advanced version of G Suite, has a new feature that stores your email headers in BigQuery, a centralized database. We use this feature to power Email Meter Enterprise.

So this might be the best time to upgrade, if you haven’t already. Here are some of the other features available in G Suite Enterprise:

  1. Scan images for specific text or confidential data: In both Gmail and Google Drive, automatic OCR will prevent and control critical data sharing.
  2. Automatic email scanning: prevent unwanted data sharing by also setting easy rules for email.
  3. Get the most out of Google Meet: the Google video-call solution will allow you to record calls into Google Drive, unlimited participants and international phone dial-in.
  4. Enforce Google Drive and Gmail security: set rules that trigger when suspicious activity happens, and get access to the new Security Center.
  5. Encrypt your email: using S/MIME encryption.

More information about G Suite Enterprise at https://gsuite.google.com/compare-editions.

All Email Received List

A detailed list can help you track emails, just by searching in real time. And this is just a work in progress!

 


Sounds interesting? For more information and next steps, head over to https://emailmeter.com/enterprise or get in touch with our sales team.

 

Gmail logs + BigQuery integration: 1 year later

It’s been a year since Google announced some new exciting features for G Suite Enterprise, including a small gem for companies that want to get the most out of their data: Gmail logs integration in BigQuery. Google stated this in February 2017:

 Today, we are making it easier to analyze Gmail logs with a preconfigured BigQuery integration so administrators can run sophisticated, high-performing custom queries, analyze their data and build custom dashboards.

BigQuery + Gmail logs

That sounds great! At Email Meter, we’re always researching and analyzing ways to bring you new powerful statistics, so this is rather interesting! We’ve been doing tests for a while. Let’s see:

[warning: tech stuff ahead]


How it works:

  • Gmail logs are stored in realtime into a preconfigured BigQuery integration with the following schema. (NOTE: the logs start after you check the option in G Suite Enterprise Administrator, read more here)
  • BigQuery organizes the data in daily tables.
  • You can run queries directly on the BigQuery console or write your own app. There are some libraries out there for different languages.
  • Queries should be made using SQL Standard, although Google uses their own SQL (“Legacy”) at their examples.

Aaaaand, that’s all… In fact, that’s all the info you can find on the internet about this. For real. No one seems to have published any info about this, or shared any code, apart from these two posts in this blog (thanks Mannetje!) – Link + Link

Querying – getting real stats!

We ran a query over the whole domain, for received emails. Each action that occurs when Gmail is receiving an email, logs a single entry in BigQuery, so we must filter a little bit.

/* received emails */
(message_info.destination.service == 'gmail-ui'
  AND message_info.source.service == 'smtp-inbound')
/* we only need the final entry + not spam */
  AND message_info.action_type == 3
  AND message_info.is_spam == FALSE

Wildcards are permitted here so we’re checking all the daily tables for the last 7 days.

BigQuery Gmail logs query example

BigQuery Gmail Logs Results Query example Gmailmeter

Voilá! We got 4713 emails, each one with extensive data. We turned some of this data into a nice email report for one single day, and used some processing to throw reply time right into the mix. Also, we generated the report for the whole domain and also for one user. Useful, isn’t it?

Gmailmeter BigQuery integration Gmail test report

Once we’re up and running, we can build reports like this one. Or complex dashboards! Privacy matters, this isn’t real data.

Response time:

At our tests, minimum response time was above 1s, reaching peaks of 5-6s or higher. This is due to Google’s BigQuery inner processes (it kind of scalates with your needs)

That might not seem impressive at first, but think of a really big company (+2000 employees) and 1 year data of email. That’s a lot of emails. Well, it still should take a few seconds for most queries! Now we’re talking. Google BigQuery is scalable, flexible and fast even for GBs of data.

Other findings:

Throughout our research we discovered some stuff that wasn’t well documented at the Schema, and we also found some data limitations (IE: it doesn’t log thread ID)

There’s also a lot of data available in the logs about deliverability, server spam rules, and email behaviour.

Gmailmeter BigQuery gmail logs

Using Gmail logs + BigQuery, we can apply Big Data techniques to email usage stats


Conclussions on integrating Gmail logs into BigQuery

Google released a year ago a feature for G Suite Enterprise that allow administrators to automatically store Gmail logs in BigQuery.

It seems that big companies are already using this feature, and BigQuery is already a established product, so the future for this product is definitely exciting.

Good news: if your company is using G Suite Enterprise already, or is thinking about upgrading, you’re just one check away from using Big Data techniques to analyze and get stats for your email usage data using a cloud-powered database. It’s powerful, fast and cheap.

Not so good news: there’s almost no info publicly available about how to build apps around this, and Google documentation is incomplete. Intensive research and development is needed.

Are you interested? Any technical details to share or ask regarding BigQuery? Let’s talk more (hello@emailmeter.com).