Materiality assessments are exploding in popularity — not only due to new legal requirements from 2024 onwards, but also because of the many academic studies that have found links between materiality assessments and superior financial performance for the companies undertaking them.

Materiality assessments are not difficult to administer yourself. The various free guides you’ll find on the internet will try to convince you that materiality assessments are complex and time-consuming processes. In reality, these guides purposely exclude any information about how to actually measure materiality so that you’ll be forced to pay them >$100,000/year to conduct these assessments for you.

This guide is the opposite. I don’t run an ESG consultancy. I’ve personally helped design numerous materiality assessments. I know what the “secret sauce” is and I’ve got no reason not to share it with you. This blog post covers everything you need to know about running a materiality assessment (or a double materiality assessment — I’ll explain the difference!). In particular, I dive deeeeeep into the details in two key areas:

1. How to measure a materiality score for each of your topics/issues → skip here

2. How to turn your scores into a materiality matrix scatterplot visual → skip here

What is a Materiality Assessment?

Materiality means that something is significant and could influence an organization’s decisions or outcomes. Traditionally, materiality was focused on how much an issue or event could financially affect a company's performance or stakeholders' decisions.

In recent years, a new type of materiality assessment called a Double Materiality Assessment has become increasingly popular. Unlike traditional materiality assessments that focus on financial performance, Double Materiality Assessments determine how a company’s decisions impact not only its financial bottom line but also broader societal and environmental issues.

Double Materiality Assessments are a common risk-management assessment used in ESG processes. ESG, which stands for “Environmental, Social, and Governance,” is a set of standards that investors use to screen investments for their potential impact on the world.

Materiality assessments have been popular in part because of past studies showing that companies carrying out these assessments have better financial outcomes, such as a 4.6% increase in market value (MIT 2021), superior profitability (McKinsey 2020), and better market results compared to competitors (Harvard Business School 2015).

Materiality Assessments Are Becoming Legal Requirements Globally

However, it wasn’t until early 2022 when governments began including Double Materiality Assessments as mandatory practices in new regulations such as the CSRD that interest began to explode exponentially:

In January 2021, the EU Commission announced the Corporate Sustainability Reporting Directive (CSRD), a new regulation that will begin taking effect in 2025 as part of the European Green Deal. As a result of CSRD, over 49,000 companies in the EU — equivalent to +75% of all company revenue generated in the EU — are now required double materiality assessments that include environmental, social, and governance issues, as well as financial impacts.

While companies that do not operate in the EU are not impacted by this, materiality assessments are already commonplace in other markets around the world, such as the SEC’s Climate Disclosure Rule in the US, which requires a traditional financial materiality assessment to be conducted (this will likely be expanded to cover double materiality in the future too).

So… How Do I Actually Run A Materiality Assessment?

If you’re looking for some not-very-light reading, the European Financial Reporting Advisory Group (EFRAG) published a +21,000-word guide to double-materiality assessments. KPMG’s 5000-word guide says nothing about how to quantify materiality, and Deloitte’s guide has hilariously vague suggestions like “use AI technology to collect and evaluate stakeholder inputs”, “conduct impact and financial assessment”, and “validate your results”. Like, woooow thank you Deloitte, that really is everything I needed to know to get the job done 🤡👍

If you’re thinking of outsourcing this work, be prepared for a hefty price tag — freelance ESG advisors charge $120,000 to $250,000 per materiality assessment, and it’s not even worth thinking about how much you’d be quoted by multinational consulting firms.

The craziest part of all is that this is not a one-time thing; Double Materiality Assessments are here to stay. Large companies should expect to conduct these assessments on an annual basis going forward, indefinitely. The better (and cheaper) your assessment process is today, the more successful you’ll be at rerunning them in the future.

The 7-Step Double Materiality Assessment Process

These seven steps are like an abridged version of KPMG’s guide to materiality assessments, with my own changes added to make it much easier to understand and implement yourself. Later in this guide, I go into much more depth on steps 4 and 5 with actionable advice, specific methods and tools to use, and real examples to illustrate the approach suggested.

Let’s look at a quick overview of all seven steps that go into creating a Double Materiality Assessment…

Step 1: Define

• What’s the objective of running this materiality assessment? It may be mandatory due to legal regulations like CSRD, to refine your sustainability strategy, or to inform strategy more broadly.

• Who will your assessment engage? For regulatory processes, this must include a broad range of stakeholders (eg. leadership team, employees, suppliers, and customers).

• What parts of the organization are you studying? Are you looking at a specific department, business unit, geographical region, or the entire organization?

Step 2: Brainstorm

• Create a list of topics that could be considered “material”. ChatGPT is your friend for this step ;)

• Look for any materiality assessments your organization conducted in the past so that you can use their list of topics as a starting point.

• Check for public documents that provide sample topic lists for your specific industry, such as those shared by the International Financial Reporting Standards Foundation or the Sustainability Accounting Standards Board, or guideline documents like this one from the European Financial Reporting Advisory Group.

• Set a time in the calendar for a brainstorming session with colleagues from various departments.

• Schedule time to engage external stakeholders via one-to-one interviews or focus groups.

Step 3: Categorize

Group topics into categories of similar issues. Generally, these categories are just the three ESG themes (Environmental, Social, and Governance), however, you can also add additional categories that are relevant to your circumstances, such as “Innovation and Technology” or “Health and Security”.

Step 4: Engage ⭐

• Create a survey with comparison-based ranking methods like Pairwise Comparison or MaxDiff Analysis to measure which topics have the highest relative importance and send it to stakeholders to collect votes.

• The survey should repeat the comparison-based ranking twice, once to gather relative importance data for financial materiality and then a second time to measure impact materiality.

• I have written more information, including interactive examples and specific suggestions for conducting these surveys, later in this article (skip ahead).

Step 5: Prioritize ⭐

Convert your financial and impact data into a materiality matrix, which will help you to identify which topics are of high importance and require mandatory disclosures. I’ve included a step-by-step process with annotated screenshots explaining how to create a materiality matrix later in this article (skip ahead).

Step 6: Approve

• Bring your materiality matrix and findings report to your leadership team to secure approval for your assessment of the company’s material topics and risks.

• Your report should cover your main findings and suggest next steps for integrating your insights into the company’s risk-management strategy.

• Make any required disclosures and reports to legal entities according to relevant regulations such as the EU’s CSRD or the US’s SEC Disclosures.

Step 7: Reflect

• Distribute your report within your company, ensuring that stakeholder groups you engaged throughout the process receive a copy of the findings they contributed to creating.

• Evaluate the assessment process and outcome to identify the areas that could be improved next time around.

• Agree to a schedule for when the next materiality assessment should be performed. Large companies that identify a significant range of high-risk topics with material exposure or organizations working in sectors with rapidly changing macroeconomic variables should be performing Double Materiality Assessments at least annually.

• Codify the steps you undertook while completing your assessment in an official document so that future assessments can benefit from your work, enabling lower-cost and higher-efficiency Double Materiality Assessments to be carried out again at regular intervals over the coming years.

There are plenty of other guides you can find online that use tens of thousands of words explaining these seven steps, however, the most neglected and under-explained parts are Step 4: Engage (how to measure the materiality or risk score for each of your topics) and Step 5: Prioritize (how to turn your scores into a “materiality matrix” scatterplot chart). I’ll spend the rest of this article explaining these two steps in much greater detail…

How To Measure Materiality For Double Materiality Assessments

By the time you reach Step 4: Engage, you’ve already got a list of potential topics grouped under 3-5 categories. Now it’s time to take this list of topics to stakeholders and quantify a score that represents their risk level or “materiality”.

Double Materiality Assessment: Measurement Methods

Most guides recommend one of two approaches for quantifying materiality topics — either a high-medium-low scale or a 1-5 star rating. Both of these methods are fundamentally flawed for materiality assessments and should never be used.

Rating scales suffer from a problem called straightlining where most respondents assign the same high rating to every topic. This is especially true in materiality assessments where every topic has already been qualified as a potential macroeconomic risk to the company. By using an overly simplistic scale like this, your output scores will end up being overly influenced by a tiny minority of respondents who assigned outlier low scores to topics.

For example, imagine asking a bunch of tourists to rank a list of summer holiday destinations. Using a 1-5 star rating scale, most respondents assign 5/5 stars to places like Spain, Portugal, Italy, Greece, Malta, and Croatia, making it very difficult to figure out their preferences.

Instead, we should switch away from an absolute measure (like a 1-5 rating scale) and use a relative measure instead, like comparing topics against each other to find their relative importance.

For example, we could instead show respondents the same list of countries but this time as a series of head-to-head votes, such as Spain -v- Portugal, Italy -v- Croatia, Portugal -v- Italy, etc. Rather than trying to rank everything at once, these head-to-head pairs have the benefit of making subjective voting much easier for respondents to complete. Afterwards, we can measure the percentage of pair votes that each option wins and use that to rank them from highest to lowest relative importance.

Here are two comparison-based survey methods that are ideally suited to Double Materiality Assessments:

1. Pairwise Comparison

Pairwise Comparison takes your list of topics and turns them into a series of head-to-head “pair votes”. Each pair vote asks the respondent to pick the more impactful topic. Once the survey is complete, topics are assigned a score from 0-100 that represents the percentage of pair votes that they won. OpinionX is a free tool for pairwise comparison surveys.

2. MaxDiff Analysis

MaxDiff Analysis is very similar to pairwise comparison, but instead of showing just two options per vote, it shows 3-6 and asks respondents to identify the most and least impactful each time. Each topic is assigned a score from -100 to +100 using the formula (#wins – #loses) / #appearances. For example, if it appeared in 20 votes where it was picked as most impactful 14 times and least impactful 2 times, its score would be (14-2)/20 = 60. You can create MaxDiff surveys with unlimited options and unlimited respondents on OpinionX.

A third option that also uses forced comparison to measure relative importance is Ranked Choice Voting, however, I would not recommend using this for Double Materiality Assessments. Ranked Choice Voting is not well suited to ranking lists of 10 or more options at a time and it is particularly challenging to rank a long list of subjective options using this method.

Here are some helpful criteria to help you choose between Pairwise Comparison and MaxDiff Analysis:

i. Individual vs Aggregate Results

What level of results do you need? MaxDiff has a higher information density, as each voting set can include up to 6 topics at a time and each vote collects both a +1 and -1 data point. On the other hand, Pairwise Comparison only shows two options at a time, meaning it requires many more votes per person to gather the same amount of data as MaxDiff Analysis. This means MaxDiff is ideally suited to generating more accurate rankings for each individual person (eg. for follow-up qualitative interviews), whereas Pairwise Comparison is better suited if you are aggregating everyone’s votes into overall results or grouping respondents’ votes by stakeholder type.

ii. Complex vs Simple Options

How long are your ranking options? If your ranking options include a lot of text or long explanations, then MaxDiff’s 6 topics per vote will require a LOT of work for participants to read and compare options. In these cases, complex statements are better suited to Pairwise Comparison.

Double Materiality Assessment: Survey Design

Now that we’ve learned about the methods that we can use to quantify the impact of our materiality assessment topics, it’s time to figure out how we build our full survey. Whether you choose Pairwise Comparison or MaxDiff Analysis for your assessment, you’ll need to use the same method twice; once for Financial Materiality and then for Impact Materiality.

First, for Financial Materiality (how external factors could financially affect the company), here are some example questions or voting prompts we could use for a MaxDiff Analysis survey:

1. "Which of the following factors do you believe could have the most significant impact on our company's financial performance in the next three years? Please select the most and least impactful."

2. "From the following topics, which do you think presents the highest and lowest risk to our company's financial stability over the next five years?"

3. "Considering our industry and market position, which of these factors do you believe could most and least influence our financial outcomes over the next decade?"

Try it for yourself with this interactive example of a maxdiff survey for measuring financial materiality:

⬆️ Created using the “Best/Worst Rank” survey format on OpinionX ⬆️

Here are some similar examples for Impact Materiality (how internal actions could affect external stakeholders, the environment, or society at large) for a Pairwise Comparison survey:

1. "Considering our company's influence on society and the environment, which of these two issues do you believe carries a higher responsibility for us to address?"

2. "Which of these issues do you believe will have a greater impact on the environment and society due to our company's operations?"

3. "Of the two topics below, which one do you think has a more significant impact on stakeholders and should be prioritized in our sustainability strategy?"

Try out an interactive example of a pairwise comparison survey for measuring Impact Materiality:

⬆️ Created using the “Pair Rank” survey format on OpinionX ⬆️

Both question types are available for free on OpinionX surveys. When setting them up, you can use OpinionX’s built-in calculator to help you figure out the right design variables, such as the number of votes per person to produce a robust dataset.

Double Materiality Assessment: Enrichment Data

I personally recommend including these three questions in your Double Materiality Assessment survey after respondents have completed the maxdiff/pairwise ranking exercises:

1. Participant Identifier

Collect your respondent’s name, email address, or some other type of identifiable data so that you can ensure there are no duplicates in your results. This also allows you to view someone’s personal ranked results, which is ideal material for follow-up qualitative interviews that aim to understand why people ranked the materiality topics the way they did.

2. Demographic Categories

Include some simple multiple-choice questions so that respondents can bucket themselves into their correct stakeholder group (for example by department, seniority, or region). This will allow you to later filter your results to look at individual segments in isolation or to compare the differences in ranked scores between different stakeholder groups.

3. Open Response

It’s possible you missed some important issues while drafting your list of potentially material topics. I usually include an open response textbox after each of the two ranking exercises so that respondents can submit ideas for additional topics they think should’ve been included in the voting list. On OpinionX, these responses can be edited and added to your voting list with just two clicks.

By including these additional question types after your financial and impact ranking exercises, you’ll have all the data you need later on to turn your survey results into a materiality matrix and generate a list of data-driven risk management suggestions using real quantitative data.

Double Materiality Assessment: Interpreting Materiality Scores

Once stakeholders complete your survey, what will the results look like?

If you used the MaxDiff Analysis voting method, your results will list each option ranked by their score, which is calculated using the formula (#wins – #loses) / #appearances. For pairwise comparison surveys, your materiality assessment results will look a little bit simpler — the options will be ranked with a score from 0-100 representing the percentage of pair votes that option won.

You can also view results for all participants at the same time using the Participants Tab on OpinionX, which shows everyone’s results in one table where each row represents one respondent.

Deciding what is or isn’t a material issue is left to each company to decide. There is no objective benchmark; companies are left to develop their own thresholds and justifications. Most large companies already have defined measures for financial materiality, but if you don’t then a good way to determine which issues are high, medium and low risk is to create a Materiality Matrix scatterplot chart.

How To Create A Materiality Matrix (Labeled Scatterplot Chart)

The final step I’ll be focusing on in this guide explains how to convert your Double Materiality Assessment survey results into a labeled scatterplot. Here’s what my end result looked like using Google Sheets and PowerPoint:

Here are the steps I followed to create this Materiality Matrix…

I copied my survey scores and pasted them into Google Sheets as a simple three-column table with the following headings: (A) Issue, (B) Financial Score, (C) Impact Score. The example below uses MaxDiff scores, hence why the scale goes into negative numbers (MaxDiff scoring is on a range from -100 to +100). I then highlighted this table, clicked “Insert: Chart” and chose “Scatter Chart” from the Chart Type dropdown:

Unfortunately, there’s no good scatterplot format that includes labels on either Google Sheets or Excel, so I had to use a manual workaround. First, I made the background transparent by going to Customize → Chart Style→ Background Color → None.

Next, I downloaded this as an image by clicking the triple dot on the top-right corner of the chart, then “Download Chart” → “PNG Image”, and opened that image in a new PowerPoint document. Once I had it opened in PowerPoint, I added three rectangles using the “Insert: Shape” tool for my red high-risk zone (#FAD6D6), yellow medium-risk zone (#FAEBCC), and green low-risk zone (#CDE5DC). Each time I added a rectangle, I removed the border, added the fill color, and clicked the “Send Backward” button to move it into the background behind the transparent scatterplot. The high-risk red rectangle should meet both axes at their extreme ends to show that a topic can still be considered high-risk even if it only scored highly on one of the two metrics.

Once you have added those three rectangles as your scatterplot background, you can add a rounded corner to the yellow and green boxes by clicking the shape, going to the “Shape Format” tab, opening the “Edit Shape” dropdown, clicking “Change Shape”, and choosing the third-last option (which is a rectangle with only its top-right corner rounded). Here’s how that looks:

This gives us the correct background and shape for our Materiality Matrix:

Like most things, the last 20% of customization requires 80% of the overall work. This includes:

1. Adding labels to each data point.

2. Creating a category system with different colors/shapes for issues on the matrix.

3. Tailoring the axes labels and titles to make it more visually appealing.

I added labels to each point manually using textboxes on PowerPoint and coordinated the background fill of each textbox to match the section of the scatterplot the issue landed in. To change the size, color, or shape of each individual point, just triple-click on any point on the scatterplot to open up a customization window like this:

The axes and gridlines can be edited on Google Sheets using the options that appear under the “Gridlines and Ticks” section of the Customization sidebar. I also used PowerPoint to add a symbols key on the right side of the scatterplot. Once you’ve tailored the styling to fit your needs, export it again as a PNG and replace the image you had been using on PowerPoint with the updated one. Here’s what my finished version looks like:

Here’s a second format I created using the same approach but with quarter-cut circles instead of rectangles for the background. Creating these quarter-circle shapes is quite a bit more complicated than the rectangles, but this approach does a better job of weighting the combined scores of both metrics when deciding which risk category a topic should be placed in.

Post-Research Steps for Double Materiality Assessments

Once you’ve got your materiality matrix, it’s time to move on to steps 6 and 7 — Approve and Reflect.

Take your materiality matrix along with a detailed report on your research methodology for quantifying risks to your leadership team to get approval for the process you’ve conducted. This report should ideally also include further detail on the participants of your survey, such as a breakdown of numbers in each stakeholder group consulted, transcripts or summaries of any interviews conducted, and next steps such as making mandatory disclosures for issues that your assessment defined as high risk (and remember that an issue does not have to score high in both metrics to be considered material; a high score in just one of the two metrics is justification enough).

If this was the first time a double materiality assessment was conducted in your organization, or if you changed the process significantly for your latest assessment, you should create a “best practice” document that details the steps you took while conducting your assessment. Most large organizations (along with any companies that have significant ESG impact risks) can expect to conduct double materiality assessments on an annual basis from 2024 onwards, so the more information you can preserve from your recent assessment, the more efficient future efforts will be. This conclusion period is also an ideal time to put some tentative dates into your teammates’ calendars for when you will begin planning next year’s materiality assessment.

— — —

Start building internal expertise in materiality assessments today

Double Materiality Assessments are not disappearing any time soon. From 2025 onwards, expect to hear about DMAs more and more often as they become regulatory requirements for an increasing number of companies. Rather than paying 7-figures annually to outsource this to an agency or consultancy that knows 10% as much as you do about your organization’s context, you can start building the internal experience and expertise needed to run your own assessments each year.

Create a free Pairwise Comparison survey and MaxDiff Analysis survey on OpinionX in under 5 minutes — even if you don’t launch it, you’re doing yourself a favor by learning how these measurement methods work and how they can be used for internal assessments like the Double Materiality framework.

— — —

Enjoyed this blog post? Thousands of founders and product teams subscribe to our newsletter, The Full-Stack Researcher, for actionable research advice just like this: