Why CPD measurement in AEC is broken — and what it would take to fix it
In 1934, T.S. Eliot asked three questions that nobody in his audience could answer:
“Where is the Life we have lost in living? Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?”
He was writing about modern society’s tendency to accumulate data while losing the thing that data is supposed to serve. He was not writing about CPD. He might as well have been.
Every year, thousands of AEC professionals in the UK complete their mandatory continuing professional development requirements. RIBA requires 35 hours. CIBSE requires 30. IStructE requires 30. ICE requires 30. The hours accumulate. The certificates are filed. The annual declaration is signed. And almost nobody — not the professional, not their employer, not the professional body, not the CPD provider — can tell you whether any of it changed how a single building was designed, specified, or built.
This is not a complaint. It is a measurement problem. And measurement problems, unlike attitude problems or funding problems, can be solved.
What we are actually measuring
The CPD system in UK AEC produces three types of data about professional development.
Attendance records. Hours completed, sessions attended, dates logged. This tells you that a professional was present. It tells you nothing about what happened while they were there.
Accreditation status. A provider has submitted a session description to a professional body, met the published criteria, and paid the associated fee. RIBA, CIBSE, IStructE, and ICE all operate accreditation schemes of this kind. The badge on a CPD invitation tells you that a human reviewed a description of the session. It does not tell you whether the session, as delivered, changed professional practice.
Satisfaction surveys. Where they exist at all — and in AEC CPD, they often do not — satisfaction surveys measure whether attendees enjoyed the session. A meta-analysis of 89 studies on training evaluation (Blume, Ford, Baldwin & Huang, 2010) and the earlier work it built on (Alliger, Tannenbaum, Bennett, Traver & Shotland, 1997) both found that satisfaction-level reactions are weak and inconsistent predictors of whether learning transfers to behaviour on the job.
The measurement instruments are proxies for proxies. Attendance is a proxy for exposure. Exposure is a proxy for engagement. Engagement is a proxy for learning. Learning is a proxy for changed practice. Every link in that chain is assumed, not tested.
This is not a new observation. Donald Kirkpatrick first described the problem in 1959, laying out four levels of training evaluation — Reaction, Learning, Behaviour, Results — and noting that most organisations measured only the first. His framework is still cited today precisely because the problem it named has not gone away. A 2023 blog post by a school-sector CPD platform cited Guskey’s 1997 critique of evaluation-stopping-at-satisfaction verbatim, and proposed the same solution Guskey had proposed in 1997: measure a sample of practitioners several months later and ask what changed (IRIS Connect, 2023).
The best available answer in 2023 is structurally unchanged from the best available answer in 1997. And even that best available answer does not transfer to AEC CPD. The school-sector solution requires an institution with internal coaching infrastructure, a line manager who attended the session, and a classroom to observe over time. An AEC professional attends a CPD session delivered by a manufacturer or independent provider, leaves the building, and returns to live projects where no third party can observe whether anything shifted. The school sector’s most sophisticated proposal cannot be applied to AEC CPD without a fundamentally different instrument.
Why this arrangement persists
The temptation at this point is to assign blame: to professional bodies for setting hour-based requirements, to providers for optimising for attendance, to professionals for treating CPD as a compliance activity. The temptation should be resisted. Everyone in this system is behaving rationally within the constraints of what can be measured.
Professional bodies require hours because hours are enforceable. A professional either has 30 logged or they do not. Requiring demonstrated outcomes is only enforceable if outcome measurement infrastructure exists — which it has not.
Providers optimise for attendance because attendance is measurable. Filling a room is visible. Changing a specification decision is not — at least not until the next project, at which point attribution is nearly impossible to establish. The provider who delivers a genuinely transformative session and the provider who delivers an engaging product demonstration receive identical feedback: a signed attendance sheet.
Professionals optimise for compliance. Logging hours satisfies the annual declaration. Reflecting on whether those hours changed anything is additional unpaid work with no institutional reward and no mechanism to record the output even if it occurred.
This is not institutional inertia or negligence. It is something more precise, and worth naming precisely: Donnella Meadows identified in her work on systems thinking that the most powerful intervention point in a system is its goal — not its rules, not its flows, but what the system is optimising for (Meadows, 2008). The CPD system is optimising for compliance completion, not competence growth. Changing the metric without changing the goal produces better compliance records. It does not produce better professionals.
Alfred North Whitehead, writing in 1929, put the distinction plainly: “Education is the acquisition of the art of the utilisation of knowledge.” Not the acquisition of knowledge. The acquisition of the art of using it. The CPD system measures acquisition — hours, certificates, accreditation status. It has no instrument for utilisation.
This is Goodhart’s Law applied to professional development. Marilyn Strathern’s formulation is precise: “When a measure becomes a target, it ceases to be a good measure” (Strathern, 1997, drawing on Goodhart, 1975). The hour became the target. The hour has ceased to be a good measure of professional development.
What is being lost
In September 2024, the Grenfell Tower Inquiry Phase 2 Report was published. Among its findings: a documented pattern of manufacturer-delivered CPD sessions being used to promote products whose fire performance had been misrepresented. Professionals attended sessions described as education, left with altered specification intent grounded in inaccurate information, and the measurement system — accreditation status, attendance records, satisfied participants — registered nothing unusual throughout.
This is what the proxy metric trap costs at its most extreme. The more common cost is quieter: professional practice that does not change when it should, because the feedback loop between CPD attendance and what a professional does next has never been closed.
The cognitive science of this is not obscure. Ebbinghaus’s forgetting curve work (1885) established that retention of newly learned material declines steeply without reinforcement — a finding replicated across more than a century of subsequent research and refined in the modern retrieval-practice literature (Roediger & Karpicke, 2006). A one-off CPD session with no follow-up is a learning event with a structurally high loss rate. More recent research on training sustainment — whether trained behaviours persist over time, as distinct from merely transferring to the job at all — finds that peer support, supervisor support, and organisational context explain a meaningful proportion of whether application persists at 30 and 90 days (Hughes, Zajac, Woods & Salas, 2020). The attendance record captures none of this. The satisfaction survey captures none of this. The hours accumulate; the learning does not.
There is one more finding worth stating plainly, because it is a finding rather than an assertion: a search of construction management and built-environment peer-reviewed literature for empirical studies evaluating CPD outcomes — behaviour or practice change, as distinct from satisfaction, attendance, or hours logged — returns almost nothing of comparable rigour to the general professional-education literature. Healthcare has a decades-deep outcome-measurement tradition. Aviation developed it under regulatory mandate after fatal incidents (O’Connor, Flin & Fletcher, 2002; O’Connor et al., 2008). Education has Kirkpatrick, Guskey, and a substantial peer-reviewed evaluation corpus. AEC has trade-body member surveys, unpublished dissertations, and provider marketing content asserting value without evaluation methodology. The sector has not yet been studied at the rigour applied to comparable professions. That absence is not a limitation in this argument — it is independent confirmation of it.
What better measurement would look like
The social psychology literature offers a more useful instrument than satisfaction surveys, attendance records, or accreditation status.
Icek Ajzen’s Theory of Planned Behaviour (1991) established, across decades of empirical research, that stated behavioural intention is a meaningful predictor of subsequent behaviour. A meta-analysis of 185 independent studies (Armitage & Conner, 2001) found that intention accounted for 27% of the variance in actual behaviour — not a perfect predictor, but substantially more informative than whether someone enjoyed a session or how many hours they sat through. Further research has sharpened this: forming a specific, articulated implementation intention — an if-then plan — produces a medium-to-large improvement in goal attainment compared with vague general intent (Gollwitzer & Sheeran, 2006).
The honest caveat: intention is not behaviour. Sheeran and Webb (2016) synthesised the conditions under which people form genuine intentions and still fail to act — the “inclined abstainer” problem. A medium-to-large change in intention produces only a small-to-medium change in behaviour. This gap is real, documented, and not dismissible.
It is also precisely why a single measure — intent — is insufficient, and why a follow-up is structurally necessary rather than a nice-to-have feature. The question asked immediately after attendance — what do you intend to do differently as a direct result of this session? — generates a meaningful, predictive, immediately collectable leading signal. The question asked at 30 and 90 days — did you do it? — converts that intent into evidence. The proportion of attendees who confirm they did is the Realisation Rate: the metric that closes the feedback loop the accreditation badge leaves open.
This two-stage mechanic is feasible. It requires no longitudinal follow-up to generate an initial signal. It does not depend on a line manager, a classroom, or an institutional observer. It works precisely because AEC professionals leave the building after every session — and the check-in finds them wherever they went.
Seneca, writing in the first century AD, identified the right unit of measurement without the benefit of Ajzen or Kirkpatrick: “Non refert quam multos libros habeas, sed quam bonos” — it matters not how many books you have, but how good they are. Volume is the wrong metric. What the books — or the CPD sessions — actually produce is the right one.
The gap is not inevitable
The measurement gap in AEC CPD persists not because the problem is insoluble but because no independent infrastructure for capturing the right data has existed.
Professional bodies cannot build it: they have neither the independence from providers nor the instrument. Providers cannot build it: they have a structural conflict of interest in how the results might read. Organisations commissioning CPD for their teams cannot build it: they lack cross-provider visibility. The individual professional cannot build it: they have no mechanism to compare their own pattern of CPD engagement against a wider cohort.
Independent measurement — collecting anonymous behavioural intent data from AEC professionals across providers, disciplines, and session types, and following up at 30 and 90 days — is the piece that has been missing. It did not exist. It exists now.
The first year of structured data will tell the sector things it has never been able to see: which session types reliably produce practice change intent, which disciplines show the largest gap between intent and realisation, and — for the first time — which specific CPD sessions are worth the hours they consume.
Eliot’s question was rhetorical. The answer to the knowledge we have lost in information, and the wisdom we have lost in knowledge, is not nostalgia. It is better measurement.
References
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T
Alliger, G.M., Tannenbaum, S.I., Bennett, W., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341–358.
Armitage, C.J., & Conner, M. (2001). Efficacy of the theory of planned behaviour: A meta-analytic review. British Journal of Social Psychology, 40(4), 471–499. https://doi.org/10.1348/014466601164939
Baldwin, T.T., & Ford, J.K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63–105.
Blume, B.D., Ford, J.K., Baldwin, T.T., & Huang, J.L. (2010). Transfer of training: A meta-analytic review. Journal of Management, 36(4), 1065–1105. https://doi.org/10.1177/0149206309352880
Ebbinghaus, H. (1885/1913). Memory: A contribution to experimental psychology. Teachers College, Columbia University. (Original work published 1885 as Über das Gedächtnis. Leipzig: Duncker and Humblot.)
Eliot, T.S. (1934). The Rock: A pageant play. Faber and Faber.
Gollwitzer, P.M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in Experimental Social Psychology, 38, 69–119.
Goodhart, C.A.E. (1975). Problems of monetary management: The U.K. experience. In A.S. Courakis (Ed.), Inflation, depression, and economic policy in the West. Mansell.
Grenfell Tower Inquiry (2024). Phase 2 Report. HC 1 (2024–25). https://www.grenfelltowerinquiry.org.uk/
Hughes, A.M., Zajac, S., Woods, A.L., & Salas, E. (2020). The role of work environment in training sustainment: A meta-analysis. Human Factors, 62(1), 166–183. https://doi.org/10.1177/0018720819845988
IRIS Connect (2023). How can you measure the impact of CPD? https://blog.irisconnect.com/uk/community/blog/how-can-you-measure-the-impact-of-cpd/
Kirkpatrick, D.L. (1994). Evaluating training programs: The four levels. Berrett-Koehler.
Meadows, D.H. (2008). Thinking in systems: A primer. Chelsea Green Publishing. (Edited by D. Wright)
O’Connor, P., Flin, R., & Fletcher, G. (2002). Methods used to evaluate the effectiveness of CRM training: A literature review. Journal of Human Factors and Aerospace Safety, 2, 217–234.
O’Connor, P., Campbell, J., Newon, J., Melton, J., Salas, E., & Wilson, K.A. (2008). Crew resource management training effectiveness: A meta-analysis and some critical needs. International Journal of Aviation Psychology, 18(4), 353–368.
Roediger, H.L., & Karpicke, J.D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
Seneca, L.A. (c. 65 AD). Epistulae Morales ad Lucilium, Letter 45, §1.
Sheeran, P., & Webb, T.L. (2016). The intention–behavior gap. Social and Personality Psychology Compass, 10(9), 503–518. https://doi.org/10.1111/spc3.12265
Strathern, M. (1997). ‘Improving ratings’: Audit in the British University system. European Review, 5(3), 305–321.
Whitehead, A.N. (1929). The aims of education and other essays. Macmillan.
FAQs
Q1: Does CPD actually improve professional practice? A1: The research base for this question is thinner than most professionals assume — particularly in AEC. A review of construction management and built-environment peer-reviewed literature for empirical CPD outcome studies (behaviour or practice change, as distinct from satisfaction or hours logged) returns almost nothing of comparable rigour to the professional-education literature in healthcare, education, or aviation. In sectors where outcome measurement does exist, the evidence consistently shows that CPD quality, not quantity, is the determinant of practice change. Hours logged are a poor proxy for practice changed. No UK AEC professional body currently requires any evidence that CPD produced an outcome beyond attendance.
Q2: Why don’t professional bodies require evidence that CPD actually works? A2: Professional bodies require CPD hours because hours are enforceable. A professional either has the required hours logged or they do not. Requiring demonstrated outcomes is only enforceable if outcome measurement infrastructure exists — and that infrastructure has not existed for AEC CPD. This is not negligence: it is a rational response to the measurement tools available. CPD Impact is building the infrastructure that makes outcome measurement possible, independently, across providers and disciplines, for the first time.
Q3: How can you measure whether CPD actually changed professional practice? A3: The most reliable near-term indicator is behavioural intent, measured immediately after attendance. A meta-analysis of 185 studies on the Theory of Planned Behaviour (Armitage & Conner, 2001) found that stated intention accounts for 27% of the variance in subsequent behaviour — substantially more informative than satisfaction ratings or hours logged. The Realisation Rate — tracking whether that stated intent was acted upon at 30 and 90 days — converts the leading indicator into longitudinal evidence of practice change. Both are captured by CPD Impact, independently of the CPD provider, without requiring a line manager, an institutional observer, or any infrastructure the professional does not already have.
