Publishing our decisions: an evidence based conclusion

Consultation response: November 2011

Foreword: Elizabeth France, Chair of the Office for Legal Complaints

The decision of what information about decisions that should be published is for the Office for Legal Complaints (OLC). We have given the issue serious consideration - and taken careful account of the consultation responses we received. Publishing Ombudsman decisions that include the names of lawyers or law firms must be evidence-based, using clear criteria. I know there are some who have felt frustrated by our process. But I am confident that the conclusion we have reached is all the better for the time we have taken to consult widely, to hear different views, discuss with government the role of Ombudsman in its current priorities, and finally, and most critically, to weigh carefully the evidence we had available from a live, operational Ombudsman service. The result is a conclusion that it is right to publish the names of lawyers in some specific circumstances, in line with the powers given to the OLC under the Legal Services Act 2007.

I am grateful to all those who participated in our consultation and who helped our approach to making this decision. What we heard very clearly from that process was a clear consensus - that it is in the public interest to provide information that includes the identity of lawyers or law firms in certain circumstances. We were also persuaded by those voices – from lawyers and consumer groups alike – who urged us to avoid forms of publication that might have a disproportionate impact on the profession - either by publishing too small a pool of names and thereby stigmatising those named, or by publishing so much as to make the exercise
meaningless.

We were very aware that, whatever decision we came to, we would not satisfy all expectations. We were also very clear, however, that we must act in line with our role as the board of the Legal Ombudsman and our obligations to fulfil the regulatory objectives of the Legal Services Act. We have therefore tried to strike a balance between the differing objectives of protecting and promoting the consumer interest, on the one hand, and encouraging an independent, strong, diverse and effective legal profession on the other.

The context for this has changed during the course of this consultation process. We looked at good practice in other Ombudsman schemes, and officials from the Department of Business, Innovation and Skills have been very helpful in clarifying the expectation from government that Ombudsmen generally should publish their data. This is in line with a presumption that, while respecting and protecting the privacy of complainants, all public bodies should be open and transparent in the information held as part of an overall approach to promoting consumer protection. I understand other Ombudsmen have been undertaking a similar process and are reviewing their approach in this area.

The OLC deliberated carefully over the options and the associated risks and benefits of the different courses open to us. The approach we have agreed is based on evidence generated through research, extensive consultation and analysis of data gathered since the scheme went live in October 2010. We consider that it also strikes the best balance between our aim of being open and transparent as well as fair and proportionate in everything we do.

This paper sets out the basis for our decision and provides more detail about the information we shall publish and when we will do so.

  1. Introduction
  2. What was decided
  3. How this will be implemented
  4. The guiding principles
  5. Outcome of the consultation
  6. What the data said
  7. Monitoring and review

1. Introduction

The Legal Services Act allows the Office for Legal Complaints (OLC) to publish reports of our investigations and decisions. In September 2010, the OLC published a discussion paper asking what sort of information it should publish. This was followed in April 2011 by the consultation paper "Publishing our decisions: an evidence based approach‟. This document represented the formal response to the first phase of the consultation and outlined the approach that the OLC would take to publishing information about Ombudsman decisions. It asked respondents to comment on the approach and on the criteria that the OLC should use in deciding if and
when to publish named case details. The consultation was supported by two events held in London and Birmingham.

The consultation document outlined a staged approach to publishing. Stage one was the immediate publishing of anonymous case studies that are based on real life cases. Stage two, from June 2011, involved publishing anonymous summaries of all cases that are resolved formally by an Ombudsman decision.

The third stage of the approach was the tracking of Ombudsman data to inform a decision about whether the OLC should adopt a policy to publish
information that identifies law firms and individual lawyers. This document describes our conclusion, based on the results of the consultation and data tracking.

2. What was decided

The OLC decided to approach this decision in three stages. We began by publishing general, anonymised, case studies. These helped everyone to learn early on what the new Legal Ombudsman was doing. The second stage was to begin to publish anonymised summaries of all Ombudsman decisions.

One of the key aims of the Legal Ombudsman is to share evidence from complaints in order to drive up standards across the profession and share lessons, thereby preventing more complaints arising. Publishing narrative summaries provides an important insight into the nature of the complaints we resolve. This provides access to a pool of case studies that will improve understanding of how the Ombudsman scheme works. It also equips lawyers and law firms with valuable learning about how to improve their quality systems and reduce the risks to receiving such complaints in future.

This third stage builds on the first two. The OLC concluded that it is appropriate to begin identifying lawyers or law firms (to add to the original narrative form of publication) in two ways from April 2012. The first information will be published in July 2012 as follows

  1. From April 2012, the OLC will commence a policy of identifying lawyers or law firms which have been involved in cases where there is a pattern of complaints or set of individual circumstances that indicate it is in the public interest that the firm or individual should be named. This information will be published immediately it has been agreed that publication is warranted, from April 2012. It will also be included in quarterly information updates published from July 2012.
  2. From April 2012, the OLC will require the names of all lawyers or law firms involved in complaints that have been resolved by a formal Ombudsman decision to be collated. This information will then be published quarterly, starting in July 2012 – in the form of a table summarising the numbers, outcomes and areas of law
    involved in the relevant cases.

The reasons for deciding on this approach are set out here.

Our first conclusion: From April 2012, the OLC will begin identifying lawyers or law firms which have been involved in cases where there is a pattern of complaints or set of individual circumstances that indicate it is in the public interest that the firm or individual should be named. This information will be published immediately after it has been agreed that publication is warranted, from April 2012. It will also be included in quarterly information updates from July 2012.

Reasons for this conclusion: The consultation showed that there is broad agreement, across consumer bodies, lawyers and professional bodies, that the naming of lawyers or law firms should be used to address severe and systematic service failures where it is in the public interest to do so. The publication of this information is important from a consumer protection perspective, even where the relevant regulator has commenced disciplinary action (which may well be over a longer
timescale). It will also expose significant service failures and drive improvements across the legal profession.

The data tracking undertaken over the past months has shown that the number of lawyers or firms identified for this reason will be very low. Clearly, identification of these individuals and firms runs the risk of some commercial detriment to them. But the OLC is confident that, by looking at each case in this category individually, it will be able to make sure that the impact on those whose names we publish is proportionate to the risk posed to consumers had we failed to do so.

How this will happen: The decision to publish details of lawyers or law firms where there has been an exceptional or severe service failing will be made by the OLC on a case by case basis. The OLC will have regard to the public interest in determining whether or not to publish. Relevant criteria may include whether:

  • there is evidence in the case of systematic failures that indicate that other consumers will be adversely affected;
  • the facts of the case show exceptional or severe impact on an individual complainant (or group of complainants);
  • there is evidence of very serious service failure; or
  • there is evidence of significant lack of cooperation with the Ombudsman that has caused detriment to the consumer in delaying the resolution of the case

If the OLC concludes that it is in the public interest to do so, the name of the lawyer or law firms involved and a comprehensive summary of the nature of the complaint and the outcome of the Ombudsman's investigation will then be published. The identity of the complainant will be fully protected, as required by the Legal Services Act 2007. Both parties will be provided with an advance copy of the text to be published, and a summary of the reasons, for their information.

Our second conclusion: From April 2012, the OLC will require the names of all lawyers or law firms involved in complaints that have been resolved by a formal Ombudsman decision to be collated. This information will then be published quarterly, starting in July 2012 – in the form of a table summarising the numbers, outcomes and areas of law involved in the relevant cases.

Reasons for this conclusion: This conclusion is based on the evidence from tracking our data, the issues raised by different stakeholders during the consultation, and government policy as set out in the Consumer Empowerment Strategy. The OLC agrees with stakeholders that the presumption should be in favour of openness and transparency. Providing information from the Ombudsman must meet the principles that we set out to guide our decision (see below). This includes publishing
information about Ombudsman decisions that will help improve standards across the profession. This also reflects the clear guidance we have had
from government about the principles it expects public bodies to apply in their approach to such matters.

In making this decision, the OLC had two key areas to consider: first, the scope of the information that should be published, and secondly, the scale of information that should be published.

Publishing statistical information about formal Ombudsman decisions that includes the names of lawyers or law firms allows for a form of presentation that offers complex information in an accessible way. It also allows contextual information about the cases to be included to help a more rounded understanding of the data. We shall present information on the number of formal Ombudsman decisions that have included an order for a remedy, and those where no remedy was awarded, and this will enable readers to draw their own conclusions, based on the data. Presenting information by area of law and including information about outcomes will also allow complaints to be put in a broader context. This may help consumers understand more about the nature of the work (and associated risks) of a lawyer or firm, allow lawyers and firms to see patterns across different areas of law, and let everyone see changing trends over time.

This form of publication has the advantage that it does not risk the inadvertent naming of complainants. The Legal Services Act 2007 expressly prohibits naming complainants. In publishing anonymised case summaries (as we have begun to do under stages one and two of this process), this risk is minimised because the summary nature of the narrative hides specific details that may identify complainants. This would not be as possible in publishing full transcripts of all formal Ombudsman decisions.

Additionally, adopting a policy to publish the full details of an Ombudsman decision is more consistent with a commitment to improve standards (rather than to scrutinise any individual lawyer or law firm). A statistical summary approach to publishing information about Ombudsman decisions strikes a better balance. It allows a more proportionate treatment of the information that focuses primarily on improving standards across the profession in the longer term and should not be seen as punishing or "naming and shaming" individual lawyers or firms.

When considering the scale of information that should be published, the board listened to the concerns that stakeholders raised during the consultation about proportionality and impact. Stakeholders raised significant concerns that naming only a small number of law firms or lawyers would have a disproportionate and detrimental impact on those involved. The data analysis showed that setting even a low publishing threshold, such as publishing the names of those with three or more cases in any financial year, would start to limit the numbers that could perhaps be named. This presented a risk of disproportionate impact. The OLC is clear that any decision to identify lawyers or law firms is not based on a blanket principle of naming and shaming. The objective is to share information and thereby improve standards, not to impose punitive measures on an individual or small group of firms.

On this basis, the OLC concluded that adopting a broader approach was appropriate. One of the things that distinguishes Ombudsman schemes from other complaints processes is their overt commitment to raising service standards. The OLC was clear that any publication should help the profession improve its practice. Publishing all cases about all lawyers or law firms was considered likely to go beyond this remit. Limiting our approach to formal decisions, on the other hand, would be in line both with the feedback received during the consultation and with government policy about making meaningful information publicly available.


In summary, the OLC has reached a conclusion which it is satisfied avoids penalising the profession for occasional lapses while at the same time ensuring that the information published will be targeted and proportionate. The decision avoids a scenario where a small number of providers would be identified, feeding a mistaken view that publication was intended to be punitive. It also ensures that the data is easily accessible. This simple and impartial approach to publication will help consumers make better informed judgements about the legal services they use. This approach therefore balances the interests of all parties and provides an objective and proportionate way forward.

How this will happen: The names of all lawyers or law firms involved in cases that have been formally resolved will be collected, beginning from April 2012. This policy will not commence before that date, and so will not apply retrospectively. Quarterly cumulative information will be published from July 2012 and annual information from May 2013. This will be presented with firms listed alphabetically. Complaint information will be aggregated to clearly show the number of formal Ombudsman decisions that individual lawyers or firms have received, the outcomes of these decisions and the areas of law involved. Approved Regulators hold
information on the size and nature of work undertaken by lawyers and law firms, and discussions will take place to look at how far it is feasible for the OLC, or the regulators themselves, to publish this alongside the case data.

The information published will be supported by wider contextual information on the number and nature of complaints received by the Legal Ombudsman in different areas of law. The information will be clearly presented and fully searchable. It will be published for a twelve month period and updated on a quarterly basis.

3. How this will be implemented

The new publication policy will be implemented from April 2012. Over the coming months, additional stakeholder workshops will be arranged to help us consider the best ways in which to present this information. This will be critical in ensuring that the information shared is relevant and meaningful to consumers and lawyers alike.

4. The guiding principles

The Office for Legal Complaints consulted on the background and principles to inform its decision in this area. What follows is a summary of the key points that the OLC had reference to in coming to its decision.

Legal basis
The Legal Services Act 2007 allows the OLC to publish reports of investigations or Ombudsman decisions if it considers it “appropriate to do so in any particular case”. In deciding on what the OLC considered to be ”appropriate”, it was guided by the regulatory objectives in the Legal Services Act, which include:

  • protecting and promoting the public interest;
  • protecting and promoting the interests of consumers; and
  • encouraging an independent, strong, diverse and effective legal profession.

The OLC sought legal advice about the way in which we could apply section 150 of the Legal Services Act. This told us that it was good practice to consult on our policy approach, and that the OLC was able to decide on either a narrative approach determined on a case by case basis, or a broader policy-based approach.

Good practice
In addition to the regulatory objectives set out above, the OLC took into account the principles of the British and Irish Ombudsman Association (BIOA). These say that good Ombudsman schemes should:

  1. establish measures to feed back information and systematic advice;
  2. give feedback to organisations on their performances at periodic intervals;
  3. be aware of the wider public benefit that they can provide, including adding value for stakeholders such as by holding organisations to account for the ways in which they deal with people and respond to their complaints; and
  4. ensure that learning is widely spread across the sector and generally raise standards.

The wider context of this debate, and what is considered good practice in this area, has changed since the OLC began to consider the issue. Since the OLC began to consider this issue, the government has placed clear expectations on public bodies around the importance of transparency and the types of information that they should be making available. In April 2011 the government published its consumer empowerment strategy, „Better Choices, Better Deals.‟ In this paper
government sets out its approach to helping consumers get better value, better customer service and crucially better support when making choices. It calls for government organisations to "publish more of their data on consumer issues, especially complaints" to help consumers achieve better outcomes. The paper states that, "[g]overnment departments, regulated businesses and public service providers will release the complaints and performance data they own unless they have good reason not to do so." Cabinet Office (April 2011) “Better Choices, Better Deals”, page 51.

The paper also refers to the role of Ombudsman specifically, stating: "Ombudsmen and regulatory bodies can improve the effectiveness of these choice tools by publishing the wealth of complaints and performance data that they collect about businesses." 

The OLC received advice from the Department of Business, Innovation and Skills during the consultation process. It is evident that there a clear expectation that the policy adopted by the OLC to publishing complaints data should fit into this broader context. The government position was heavily cited in the consultation responses we received, particularly from consumer groups as well as bodies such as the Office of Fair Trading stating that they would like to see firm evidence to support any decision for not publishing.

The OLC’s principles
During this consultation, the OLC had reference to six principles which it took into account when deciding how to approach publishing Ombudsman decisions. These were based on the regulatory objectives and BIOA principles set out above, as well as on the organisation's own values. The OLC asked for views on these principles as part of the first stage of our consultation on this issue. As the vast majority of respondents agreed with them, the OLC continued to use them to guide its thinking. The principles are:

  • Openness
  • Being clear about how we work
  • Helping lawyers
  • Helping consumers
  • Publishing the right amount of information
  • Managing the impact on the legal profession.

5. Outcome of the consultation

During the consultation, the OLC asked for views on its proposed approach, in particular the criteria that should apply to identifying individual lawyers and law firms.

Respondents were largely supportive of the approach outlined in the consultation. Some felt that six months of data tracking would provide sufficient evidence upon which to base a decision. Conversely, a number of organisations believed the Ombudsman would gather insufficient data and intelligence in this time. They therefore felt that tracking should be extended for an additional three months.

It appears that the criteria proposed were largely in line with people's expectations, although one respondent felt that applying any criteria at all was too cautious and would restrict consumers from making informed choices. While opinions varied significantly on the criteria that should be applied to publishing, there were very few suggestions on what additional data should be tracked to inform these decisions. A number of organisations asked for more clarity, particularly with regard to any
qualitative criteria that could be applied. 


There were very strong (and opposing) views on what the basis of our eventual decision should be made. One respondent felt that:

"The OLC’s decision on whether or not to adopt a policy of identifying individual law firms should start with the broadest justifiable approach to publication, and then consider whether or not there are any specific exclusions required in response to issues that are identified..... information should only be withheld if it falls under the rules of FOI."

This echoes the principles we originally applied to this consultation. These stated that the presumption should be in favour of transparency, taking into account risks and issues to determine an approach.


However, a small number of respondents remained firmly against the principle of identifying law firms:

"We remain of the view that the identity of both consumers and lawyers using LeO’s service should not be made public."

The debate focused around the particular criteria that should be applied and whether this was best met through an objective or more subjective approach. Many felt that there should be no automatic triggers / criteria that lead to publication, with decisions made on a case by case basis. Discussion at the consultation events tended towards a preference for a more subjective approach being taken. Decisions on whether to publish should only be made once all the circumstances of the complaint and the implications of publishing had been fully considered. A number of organisations also felt that the outcomes of the nature of the complaint on the consumer should be considered when deciding whether to identify a firm.

Concerns were also raised that the adoption of a subjective method would result in fewer firms being named. The probable consequence of this was that the impact of being identified would be disproportionate, as it would be seen as more of a singling out process.

All agreed, however, that the approach needed to be proportionate, and cost-effective in its use of resources. We should be adopting a process that could be explained and that took into account a notion of "public risk" and what would be in the public interest. 

6. What the data said

"Publishing our decisions: an evidence based approach" identified seven key criteria that could be applied to publishing our cases:

  1. firms involved in cases that our Ombudsmen consider indicate an exceptionally severe degree of service failure;
  2. firms that our Ombudsmen consider have demonstrated particularly good practice in resolving a complaint;
  3. firms that our Ombudsmen consider have a very exceptionally high level of complaints, given the size and nature of the business;
  4. firms involved in complaints that are resolved formally;
  5. firms with more than three complaints where a remedy is awarded within a twelve month period;
  6. firms involved in complaints where a remedy is awarded; and
  7. firms with more than three complaints investigated by us in a twelve month period.

The data covering these conditions and outcomes has been tracked for the past six months and the consequences of publishing against each of these criteria have been analysed. This includes the number and types of firms that would be identified through the application of each criterion. 

Primarily, this analysis highlighted the importance that scale, context and proportionality would have on any in decision that we made.

Scale and context

The tracking showed that the application of a policy which is limited to naming lawyers or firms whom the Ombudsman considered had delivered an exceptionally poor service would result in very few being named. In tracking this data, the Ombudsmen applied a definition based on public risk, which generated a list of approximately six lawyers or firms that would have potentially fallen into this category.

The OLC was also mindful of the potential unintended consequences of naming. Publishing details of all cases where a remedy is required would mean the scale growing significantly, with nearly 2,000 names to be included. This did not sit well with an approach that favoured proportionality and fairness. Publishing this information would have meant it was difficult to analyse trends across the breadth of the Ombudsman's work. And it would limit publicly available information
only to those cases where a remedy was warranted. As it is possible that the Ombudsman may decide not to require a remedy, this approach did
not seem to strike a fair balance in terms of placing Ombudsman decisions in a wider context.

A number of respondents questioned the impact that publishing formal decisions could have on the resolution process. We ran an analysis of the 618 lawyers and firms that would have been identified under this criterion. This showed that there were a small number that had received a high number of complaints which had all been resolved informally and so would not have been published under this criterion.

Focusing on formal Ombudsman decisions was considered to be a fairer approach. Here, cases had been through the full, formal process of the Ombudsman scheme, and the issues raised had been thoroughly investigated. Publishing information about all cases and lawyers or law firms seemed disproportionate after having looked at the numbers of cases. Limiting our approach to formal decisions seemed to be in line with both the feedback we received during the consultation and with government policy about making meaningful information publicly available. 

Proportionality

Tracking also showed that the application of any numerical threshold would have a significant impact on the volume of individuals and firms that would be identified. For instance, the number of firms or lawyers that would be identified under criteria five and seven varied significantly, depending on the numerical criteria applied:

  • Criterion five: firms or lawyers with more than three complaints where a remedy is awarded in a twelve month period
Criteria Number of lawyers or firms Number of cases
3 or more 182 770
4 or more 80 464
5 or more 42 312
9 or more 11 129
  • Criterion seven: firms or lawyers with more than three complaints investigated by us in a twelve month period.
Criteria Number of lawyers or firms Number of cases
3 or more 393 1874
4 or more 213 1334
5 or more 116 946
9 or more 23 394

Conclusion

The OLC considered proportionality and scale both with regard to the impact on the accessibility and relevance of information provided and the reputational and professional impact of the identification of firms or lawyers. 

Looking at the data from the application of different criteria, it became clear that the choice of methodology would have an impact on which firms or lawyers might be identified. We found that no poor service had been provided by the law firm or lawyer in a significant number of cases that are resolved both at a formal and informal stage. When a threshold for numbers of cases resolved was added to this (as proposed by some respondents and noted in our consultation criteria), the risk of a
potentially disproportionate impact on some firms grew. This is because not all of the proposed criteria adequately accounted for this important factor, and the level of contextual information that would be needed to supplement this approach was high. The data also showed that there were likely to be changes in patterns of complaints over time. Adopting an approach such as publishing on a rolling quarterly basis, would provide this context to consumers and lawyers, and it would mitigate some of the risk of publication being overly punitive on individuals or firms.

Most importantly, the data tracking highlighted the sheer volume and complexity of the data that is held. The application of each criterion produced significantly different results in the nature and types of firms that would be identified, and it is clear that no one approach would adequately meet the agreed guiding principles. This highlighted the value of applying a consistent approach to publication and providing consumers with sufficient information to help them form their own view
and make judgements on the relevance and significance of the data.

7. Monitoring and review

The impact of this policy on our complaint handling will be fully monitored and evaluated. A baseline will be established from which to assess the impact of the policy on the resolution process. Finally, the approach to publishing information will be reviewed within two years of implementation.