Statistical analysis and methods for the 2019 Business Agility Report.

We worked with the Business Agility Institute to provide in depth statistical analysis of the 2018 and 2019 Business Agility Report.

The Business Agility Report looks into the current state of "whole-of-business" agility, as an extension to the movement that brought agility to software development practices.

  • The 2018 report is available here
  • The 2019 report is available here
  • To discuss the report in more detail or to arrange a demo of our MapperTM suite please contact us here. MapperTM enables organisations to get a real-time view of their most valuable assets: their people and cross-functional teams, the work they do, and the organisational structure they work within. Guided by the organisation's evolving operating principles, our tools help people leaders plan and allocate teams to adapt to the demands of the work.

    When looking at how a business operates, and which factors positively or negatively contribute to agility, we believe it's important to back claims with data. This analysis and methods section does this with the scientific rigour expected of the field.

    Business Agility Report 2019 - Analysis Results

    Validation of the Business Agility Survey

    The results of Cronbach’s alpha analysis indicated that the 2019 Business Agility Survey exhibits a very high degree of internal consistency (𝛼 = 0.97).

    Did smaller organisations have greater business agility than larger organisations?

    (BAR2019: "Company size correlates to business agility" segment)

    Hypothesis 1. Smaller organisations, defined as having 1 - 50 or 51 - 200 employees will have significantly higher agility than organisations with greater than 10,000 employees.

    Overall, average agility scores decreased as organisation size increased (1 - 50 employees = 5.45, SD = 2.25, 51 - 200 employees = 4.76, SD = 1.82, 201 - 1,000 employees = 4.12, SD = 1.53, 1,001 - 10,000 employees = 3.79, SD = 1.84, 10,000 + employees = 3.70, SD = 1.22). Partial support was found for hypothesis 1. After fdr corrections and controlling for, employee role, geographic region, and length of agility practice, we found that organisations with 1 - 50 employees scored 1.34 points higher (p = 0.001) on average total maturity than organisations with greater than 10,000 employees. Whilst no other organization size category attained fdr corrected significance, all coefficient values trended in the expected direction (0.75, 0.45, and 0.23) for increasing organization size categories (51 - 200, 201 - 1,000, 1,001 - 10,000). In addition, organizations with 51 - 200 employees approached nominal significance (p = 0.08), scoring 0.75 points higher on average total maturity than those with greater than 10,000 employees. See table 1.

    Was there a geographic trend to agility?

    (BAR2019: “Business agility around the world” segment)

    Hypothesis 2. Organizations in the United States of America and Europe will score higher on average maturity than organizations in Asia.

    Support was not found for hypothesis 2. Compared to organisations in Asia, after fdr corrections and controlling for employee role, organisation size, and length of agility practice, neither the USA or Europe scored significantly higher on average total maturity (either at nominal or fdr corrected significance). In addition, no organisations from any other geographic location scored significantly higher with mean average maturity grouped closely for all locations (USA = 4.95, SD = 2.1, Worldwide = 4.7, SD = 2.06, South America = 4.49, SD = 1.43, Asia = 4.46, SD = 1.89, Europe = 4.10, SD = 1.98, Eastern Europe = 4.00, SD = 1.76, Oceania = 3.97, SD = 1.73). See table 1.

    Were there differences between internal and external employee rated average maturity?

    (BAR 2019: “Perception of business agility” segment)

    Hypothesis 3. External partners and consultants will rate organizations significantly lower on average maturity than those in the C-suite.

    Support was found for hypothesis 3. All employees rated average maturity lower than those in the C-suite (C-suite = 5.86, SD = 2.11, LOB/division leader = 4.66, SD = 1.63, Senior executive = 4.28, SD = 1.89, Manager = 4.26, SD = 1.63, Individual contributor = 3.83, SD = 1.62, Supplier/partner/consultant = 3.81, SD = 1.83). After fdr corrections and controlling for organisation size, geographic location, and length of agility practice, individual contributors rated organisation average maturity 1.334 points lower (p = 0.003) than those in the C-suite. In addition, suppliers, partners, and consultants rated average organisation maturity 1.36 points lower (p = 0.001) than those in the C-suite. See table 1.

    Additional finding

    As we controlled for length of agility practice in our analyses, we also observed a time dependent effect for average rated maturity. Specifically, organisations that were more than 8 years into their agility practice rated 2.036 points higher (p = 0.001) on average maturity compared to those who were less than one year into their agility practice. See table 1.

    Self reported agility: Cross-validated exploratory regression analyses

    (BAR 2019: “Key Findings” segment)

    For our hypotheses generating procedure on the 2018 BAS survey, we found that collective ownership (b = 0.265, p < 0.001), relentless improvement (b = 0.230, p < 0.001), funding models (b = 0.165, p = 0.002), and value streams (b = 0.287, p < 0.001) were significantly associated with self-reported agility after fdr corrections and controlling for all variables in the previous analyses. Taking this variable subset and validating on the 2019 BAS dataset, all variables except collective ownership remained significantly associated with self-reported agility. As all variables were mean centred due to multicollinearity, all unit changes are reported in standard deviation (SD) form. For every one SD increase in relentless improvement, self-reported agility scores increased by 0.202 points (p = 0.013), for every one SD increase in funding models, self-reported agility scores increased by 0.215 points (p = 0.001), and, for every one SD increase in value streams, self-reported agility scores increased by 0.328 points (p < 0.001). See table 2.

    Year on year changes in average maturity: 2018 to 2019

    (BAR 2019: “Business agility maturity” segment)

    A small increase in maturity was reported by employees that completed the survey in both 2018 (mean = 4.7, SD = 1.9) and 2019 (mean = 4.9, SD = 2.0). However, a paired samples t-test indicated no significant differences between years (t = - 0.46, p = 0.64).

    Qualitative Analysis

    Self-reported Barriers to Implementing Agility

    (BAR 2019: “Challenges along the journey” segment)

    Our thematic analysis identified a number of key themes in the data which were seen as barriers to the respondents’ organisations agility transformations. The most prevalent of these themes were:

  • Organisational alignment around business agility;
  • poor leadership;
  • difficulty separating from established ways of working;
  • understanding of what Business Agility is;
  • cultural challenges;
  • the need for a shift in mindset;
  • a lack of Agile metrics;
  • a lack of buy-in, scaling and growth issues; and
  • the amount of support for agile implementation
  • Self-reported Impacts of Implementing Agile Business Practices

    (BAR 2019: “Key Findings” and “Business agility successes” segments)

    Our thematic analysis identified a number of key themes in the data which were seen as key successes and impacts - both internal and external - which flowed from implementing an agile transformation. The most prevalent of these themes were:

  • increased transparency and collaboration within the organisation;
  • improvements in development and delivery - including faster turnaround times, higher quality offerings, and more tailored solutions;
  • increases in customer success and satisfaction - including improved relationships with customers;
  • improvements in employee engagement and satisfaction;
  • and improved success in the market - including revenue increases, market share, and brand recognition.
  • Conversely, another theme which emerged during this analysis was one of no impacts or successes following an agile transformation.

    Tables and Figures

    Table 1. Organisational determinants of average total maturity.

    FDR p-value cells shaded green remained statistically significant.

    Table 1. Organisational determinants of average total maturity.

    Table 2. Results for the 2019 self-reported agility analyses.

    Predictor variables selected from the 2018 survey using elastic net regression and cross-validated on the 2019 dataset. FDR p-value cells shaded green remained statistically significant.

    Results for the 2019 self-reported agility analyses.

    Figure 1. Analysis pipeline for the 3 multiple linear regression hypothesis testing models.

    Figure 1. Analysis pipeline for the 3 multiple linear regression hypothesis testing models.

    Figure 2. Analysis pipeline for the exploratory hypothesis generating model for each of the 26, 2019 BAS items.

    Figure 2. Analysis pipeline for the exploratory hypothesis generating model for each of the 26, 2019 BAS items.