A response to the ACOLA review 2016: HDR Supervision

The goods train Research Training

Recently, the Australian Council of Learned Academies (ACOLA) released it's Review of Australia's Research Training Scheme Final Report (2016). The Report was derived from numerous sources, including submissions from across Australia from those involved in research training, and training research supervisors.

What does the report say about supervision?

My comments here are restricted to one section of the Report - Section 10: Evaluation of supervisor
competency and performance (pp.87-93). The key finding of this section:10.5 Key finding 10, is as follows:

Universities have a responsibility to provide ongoing high quality HDR supervisory training, and a responsibility to act where supervisory performance falls below expected performance levels. Outstanding HDR supervision should be recognised and reinforced by universities through the application of professional standards and rewards for performance (p.93).

 This key finding was arrived at after considering the supervision experiences of HDR candidates (10.2); and key issues relating to improving the quality of HDR supervision (10.3). These included:

• Inconsistent or lack of training
• Structural issues affecting quality of supervision
• Unclear involvement of supervisory panels
• Lack of quality assurance mechanisms (pp.88-89).

The report then makes a number of recommendations, broadly under the rubric that research supervision needs to be professionalised. In summary:

• There needs to be supervisor training, and there should be a framework to frame this training
• There needs to be improved support for supervisors
• There needs to be consistency in the role of supervisory panels
• Supervisor performance needs to be monitored

My response to Section 10 of the Report.

1. Silos

The Key Finding, key issues and recommendations, in my mind, can be summarized as largely adding few new insights to improving supervision practices. In addition, they appear unaware of many years of scholarship about academic development that may have proved useful and fruitful to establishing a "new normal". This is the unconscious working of the Research Training Silo (yes, another meaning for RTS). I've sat in a graduate studies office for less than a year, but I'm a veteran of academic development and the pursuit of the new normal in a digital age,(eg and eg)  and the report confirmed my experience of the silo nature of "research" and "learning and teaching" in Australian higher education. It's disappointing. 

2. Digital literacies.

The ACOLA report mentions the word "digital" twice only, and one of these times is actually a job title. In the section on supervision training the word "digital" is not used. The need for digital literacies in higher education practices is a basic rule 101 in today's academic development conversation, yet it does not appear as a key issue in the report. It is a critical oversight.
[Re research training: The section on "skills development in research training" (pp. 36-40) and Table 9 (p.39: possible research-specific and transferable skills for HDR candidates) do not foreground digital literacies. Perhaps this is because no-one mentioned it in submissions?]

The question not asked: What does research training and research supervision look like in a digital age? 

The review should have stepped up and asked this question. It is a great shame other reports, (outside the Research Training Silo) such as Advancing Australia as a Digital Economy (among a great many) were not drawn upon. And given the OECD is referred to in the report as a touchstone for international comparison, surely it should also have drawn on the OECD publication OECD Digital Economy Outlook 2015.

3. Minimising.

OK, so the report does acknowledge that 1 in 5 HDR candidates are not satisfied with their supervision, and that this number is qualified by access to post graduation employment. But then the whole section self-sabotages by this important throw away paragraph (emphasis added):

The data suggests that universities have an opportunity to improve the supervisory experience and provide further support for the minority of candidates that are currently not enjoying a positive supervision experience (p. 87).

The section begins by saying "1 in 5" not satisfied, but by the end minimized to "the minority of candidates"! Sorry, but 20% is actually a very large number! It's unacceptable! It is not good enough, and it must change. It may not be the role of the report to use exclamation marks as I have, but using the word "minority" in this context, and in this way, if inappropriate and minimizes the issues. For a more detailed discussion, see the NSW Ombudsman's recent report Complaints about the Supervision of Post-Graduate Students (2016).

4. The "training" silver bullet
In Blog posts, MOOCs, twitter and reports, the silver bullet called "training" is recommended for solving the issue of supervision. Apart from the focus on "training" being very 1980s, there is no notion in the report that before any "training" is done what is needed is a broad and robust conversation, hopefully with academic development professionals, about what best practice might be in a digital age in terms of continuing professional development. This isn't just a conversation about research frameworks, it's also about learning, learning design appropriate to academic development (supervision), and where supervision fits within academic work. It's also about access and equity.
As I look around Australia, the preponderance of training in relationship to supervision appears to be off-the-job, short, one-off, compliance-focused events that foreground procedural "how to" knowledge. There are many password protected sites, and a great lack of sharing.
In the "open" (free, not password protected) there are self-directed asynchronous resources such as the Research Supervision Toolkit and Edith Cowan's Improve HDR Supervision Practice eBook Both have suggestions about local usage that assume a synchronous face-to-face mode.
Generally speaking however, resources are developed in the absence of conversations with educational designers or technologists (the Silo effect at work) so that the resources lack appropriate, well-thought-out design-based approaches to learning and changed practices. (Let alone the notion of academic development being achieved through, for example, professional learning conversations).
From a learning design perspective, throwing things online could be regarded as "first generation" distance education (slight variation - print media, but put online) if one used Taylor's taxonomy. (The sector should aspire to "fifth generation"!)

What is missing from the report?

Scary missing things

1. Professional learning theory (academic development)
A deeply embedded link needs to be forged between what we already know about academic practice, academic development and research supervision pedagogy. Reference is made to the UK Vitae Research Development Framework, but this not provide insight into how to develop academics who have the capacity to support candidates to achieve the framework. There are existing professional standards for academics, such as the Australian University Teaching Criteria, Professional development framework for teaching in higher education; the UK Professional Standards Framework, the Kinship model for teaching and learning, among many. As I'm an advocate of supervision in a digital age, I also think the following frameworks should inform any conversation about quality supervision: ACODE Benchmarks, JISC support and development materials, and the E-Learning Maturity Model.

2. Acknowledgment of cultural practices!
Broadly, the report is an expression of the cultural norms of the research training sector. "Disruption" and "transformation" are missing in action (hiding in the corner with "digital") so perhaps the sector thinks only tweaks, additions, tightening ups and changes of focus are needed? (That's not a view I share).
In terms of "supervision": Changing supervision practices is about cultural change. Cultural change cannot be wrought through Key Finding 10. For example: the word "completions" is wrapped up in, and expressive of, a range of cultural norms. It is mentioned 48 times in the report. "Completions" is a word that has big legs and assumed meanings in the research training world. [The answer to the question "How many completions do you have?" being a chaff-sifter in academic parlance.] Sadly, the report does not grapple with the meaning of "completions" as a cultural practice, and arguable it is only by so doing that supervision practices can be transformed. But this is only one example - the report itself appears unaware of it's acculturation. Surely a review of this substance should have looked at it's own limitations, and the limitations of those in the sector making submissions?

3. Frankness
The truth is there are some supervisors that are serially terrible - and institutions that are sometimes complicit in doing nothing about it. Key Finding 10 fails the frankness test when it comes to dealing with the conflicts of interest that can occur when a poor supervisor (who achieves completions) brings in a big reputation, big industry connections and large sums of research funding. We don''t know nearly enough about this occurrence, for a whole range of reasons. But for change to take place frankness is needed.
It also fails the frankness test by not mentioning complicated things like the fact that we know that bullying takes place in some industries (such as clinical settings) by senior staff who then may also supervise as Adjunct in Universities. The examples are many, here are two: 1, 2. What procedures are put in place to ensure bullying cultural practices that are "common" in external workplaces are not imported into the supervision of research students? Again, a conflict of interest exists - the disinclination to expect Adjuncts to develop as supervisors because they "supervise for free".

Not to mention the lack of clearly, upfront stated zero tolerance for bullying by supervisors of candidates. That is a whole other Blog entry, but see a tweet of mine below:

Evaluating and monitoring supervisor performance cannot be fully understood nor enacted in the absence of such frankness about conflicts of interest and an eyes-open approach that grapples with cultural norms that need to change.

4. Leadership in thinking about the changing security of academic work and implications for supervision
Although the report considers postgraduate career destinations in the context of academic careers (p.12) and generally acknowledges the decline in secure, tenured employment in universities, it only briefly flows this acknowledgment through to considering the implications for quality supervision (under "structural issues" p.89). The industrial organisation of supervisors, prescribed through policy across Australia, is predicated on secure and stable employment. The appointment of Principal Supervisor, for example, typically assumes consistency of opportunity (to demonstrate that they have brought candidates to completion, thus justifying being appointed as a supervisor) and consistency of contract (that the supervisor will be there at the start and finish of a candidates long journey). Yet HR policies and employment practices no longer guarantee either. It is a great shame that Key finding 10 did not refer in any way to this critical structural issue.
To put this colloquially: given the insecure nature of academic work is not going to change anytime soon, how on earth are #nextgen supervisors going to get to be "Principal" supervisors if individual University policy and practices aren't changed as a matter of urgency? This inter-generational "new divide" will characterise academic work into the future and needs new thinking and new solutions.

5. A connection between the report's findings in other chapters, and implications for supervision
The Report usefully highlights the need for greater focus on industry-university partnerships, internships, and much greater support for Indigenous researchers. Yet these findings don't flow through into Key Finding 10 or to thinking about the continuing professional development of supervisors in the context of a shift towards Master of Research; Internships; and Indigenous education; internationalisation and so on. For example, in the context of Indigenous education supervisors need to develop and demonstrate, cultural competence and safety. While this was noted in Section 11 in the report (Under-represented groups in HDR training) the implications for supervisors was not then noted in Section 10.

Conclusion

Although I have read the whole report, I have focused on one section - Section 10: Evaluation of supervisor competency and performance. I had hoped the report would provide substantial leadership and vision concerning the way forward, but was disappointed. The reliance by the research training community on "training" as a silver bullet is evidenced in the report, which lacks engagement with thinking about supervision as a 21st century capability in a post-industrial labour market/economy/world - in many ways discontinuous with industrial forms and norms of supervision labour. Yet it is these very norms that remain un-surfaced in the review, and therefore resistant to exposition or disruption. I can only guess that this reflects the likewise limitations of submissions made re "supervision" by the sector.