CIFellows 2020 For the Record

< Back to CIFellows 2020 for the Record Landing Page

< Back to Appendices Landing Page


Appendix E: Full Proposal

Full Proposal (pdf)


Project Summary

Overview

The current coronavirus (COVID-19) pandemic is disrupting aspects of daily life and work, including having a serious impact on the current faculty recruiting season in the computing research community. A Computing Research Association (CRA) survey completed on April 1st counted about 100 academic computing job positions being pulled from the market, while we now see universities forecasting financial losses of $10-100M for the upcoming academic year. Thus what is needed is a bridge that keeps highly trained researchers in the academic pipeline to preserve future computing innovation and to meet the training needs of future computing professionals, as these will be the backbone of the future economy. 

CRA and its Computing Community Consortium (CCC) provided such a bridge for the severe economic downturn a decade ago, using NSF funding to administer three cohorts of Computing Innovation Fellows (CIFellows). This postdoctoral project kept 127 young scholars in research with career-enhancing programs.

This CIFellows 2020 proposal takes inspiration from the original CIFellows project, but adapts it to the current uncertain situation, by allowing more flexibility, the option of doing a postdoc at the applicant’s current institution, and a significant mentoring/cohort-building component that is based on best practices that emerged from the original program. Fellows may come from any research area under the umbrella of NSF Computing and Information Science and Engineering (CISE). Fellows will engage in a 1-2 year postdoc experience that furthers their career development in new ways. Selection will be made in a “technical program committee style” with strict adherence to COIs and based on a holistic evaluation of merit and diversity along many dimensions, with major emphasis on intellectual merit and broader impacts in applicant materials.

Intellectual Merit

The principal way this proposal advances knowledge in CISE domains is through the union of the intellectual merit created by all the funded CIFellows. Based on the previous CIFellows cohorts’s performance and the selection process, one can expect many advances from such a gifted, high-energy, and well-mentored group. The proposal also lays the groundwork for a future study understanding factors in the successful launch of academic careers.

Broader Impacts

As with intellectual merit, the principal way this proposal provides benefits is through the union of the broader impacts created by all the funded CIFellows. As above, one can expect many benefits from such a gifted, high-energy, and well-mentored group. In addition, this proposal has a critical direct broader impact: retaining a cohort of early career professionals in areas under the umbrella of NSF CISE to help create the future, as well as enhancing training in areas critical to the 21st Century, thereby propagating the societal benefits further as the cohort subsequently advance in their careers.


Computing Innovation Fellows 2020 Project

1. Introduction: The Need

The current pandemic caused by Coronavirus 2019 (COVID-19) is disrupting aspects of daily life and work across all sectors, including the academic computing research community. And yet, advances that began with basic computing research are making it possible for many to work, learn, play, and remain connected, while following shelter in place and social distancing recommendations. Despite staying at home with limited direct contact outside the family, people are managing. Perhaps most directly, video conferencing tools running over the Internet have quickly become the standard for work meetings, distance learning lectures, yoga classes, and playdates for children. Fortunately, past information technology research has enabled an IT ecosystem that has proven robust to rapidly increasing and changing demand. 

Among many other things, COVID-19 is having a serious impact on the current faculty recruiting season in the computing research community. Faculty recruiting visits have been moved online or cancelled altogether. Second visits to support better decision making by candidates are not currently possible. Because the computing faculty recruiting season tends to run well into April, or beyond, most universities were not close to completing interviews when travel restrictions went into place. Beyond the challenge of interviews, COVID-19 will have an economic impact on universities, though the specifics are difficult to project at this time. Some universities have already reduced or frozen hiring for the fall to hedge against economic uncertainty. 

To understand the developing impact of COVID-19 on faculty hiring, CRA conducted a survey of Computing Research Association (CRA) member institutions. We asked about (1) their academic hiring and (2) the job searches of their finishing PhDs. The top-level conclusion from the survey is that serious impacts to academic hiring and PhD job searches are already being seen, and there is an enormous amount of uncertainty going forward. The survey was conducted between March 27 and April 1, 2020. The situation is still shifting rapidly, though clearly in the direction of more negative effects, thus this data represents a lower bound on the impact. Responses were received for 151 academic units: 119 US doctoral (87 public, 32 private), 7 Canadian doctoral, and 25 US non-doctoral. Even this early survey indicated that just short of 100 academic job positions were being pulled from the market.  With the benefit of just a few additional weeks, we now see universities forecasting financial losses in the $10-100M range for the upcoming academic year and contemplating delays to full on campus operations until 2021 (Binkley & Amy, 2020). Additional data from the survey is used to help determine the size of the proposed program.  The survey results can be found in Supplemental Documents.   

This leaves the computing research community at risk of losing a class of young researchers who cannot afford to wait out the current disruption for future hiring seasons and thus may leave the research career path permanently. It is not a stretch to anticipate that the loss of this workforce in the research pipeline will have long-lasting downstream effects on computing innovation and impact. Moreover, the COVID-19 economic downturn may result in disruption that persists beyond the next hiring cycle. And yet the demand for computing courses by majors and non-majors shows no signs of abating (Singer, 2019), as computing broadly construed (e.g., to include data science, machine learning, cybersecurity, robotics) pervades nearly all industrial sectors and government. Thus what is needed is a bridge that keeps highly trained researchers in the academic pipeline to preserve future computing innovation and to meet the training needs of future computing professionals, as these will be the backbone of the future economy. This proposal meets the need for a career-enhancing bridge through a postdoctoral fellows program and associated cohort community-building activities.

2. A Prior Success: Computing Innovation Fellows Project

While COVID-19 is unprecedented in modern times, disruption to the career trajectory of a class of computing PhDs and postdocs has some precedent. In the severe economic downturn of the 2008 time period, many universities greatly reduced faculty hiring. In response to a similar concern about the loss of computing researchers, the Computing Research Association applied for and received three NSF awards to administer three cohorts of Computing Innovation Fellows. In brief, Computing Innovation Fellows (CIFellows) paired recent PhDs with mentors in academic and industrial research labs, for a 1-2 year postdoc position. In total, 127 young scholars took part in the program, and many went on to successful academic and industrial careers, enabled by the opportunity to develop as researchers.

For example, to quote David Mimno, Associate Professor at Cornell University, 2011-2013 CIFellow, “I want to emphasize how much the chance to stay in academia, granted to me by my CIFellowship, has allowed me to give back to students. Both of my graduated PhD students have gone on to tenure-track faculty jobs…Teaching is more than a job for me, it’s a calling. I am so grateful that the CIFellows program was there for me when I needed it.”  Career stories of former CIFellows can be found in Supplemental Documents.   

The first CIFellows program underwent a formal evaluation conducted by the CRA Center for Evaluating the Research Pipeline (CERP) (Cundiff et al., 2014). CERP compared the experience and outcomes of postdocs who participated in the CIFellows program with non-CIFellow postdocs. To quote from the executive summary: “Findings suggest that CIFellows had more positive postdoc experiences and, at the time of the survey, were earning higher salaries compared to Non- fellow Postdocs. Notably, these differences in outcomes emerged even after accounting for baseline differences in merit between the two groups.” The survey also indicated opportunities for improving postdoc experiences generally. Near the end of the first CIFellows program, the CRA Board of Directors endorsed and adopted a Memo on Computer Science PostDoc Best Practices (Jones & Gianchandani, 2012), highlighting key features for successful postdoctoral experiences, many of which are followed for this new program. 

3. Building a Program for 2020

We propose to take inspiration from the original CIFellows program but adapt it to the current situation, thus providing a similar career development bridge experience for the current generation of graduating PhD students and postdocs.

3.1. Program Overview

Flexible Postdoctoral Plans. In recognition of differences between this crisis and the prior economic downturn, we propose a new CIFellows program for computing researchers who were on the academic job market, but are without a placement for the coming year. We recognize the need for flexibility given uncertainties about travel, the housing and rental markets, and the timeline for universities to welcome researchers back onto their campuses. Therefore, we deliberately allow flexibility for this round of the program: a CIFellow will engage in a 1-2 year postdoc experience that furthers their career development in new ways, under the supervision of a mentor. Examples of such arrangements could include moving to a new institution under a new mentor (the prior CIFellows model), working with a new mentor elsewhere while remote, working with a new mentor at the same institution (e.g., in an adjacent research area), or working on a distinct career-expanding project at the same institution (e.g., applying research to the circumstances of the new COVID and post-COVID world). Notably, the idea is not to support simply an extension of time in the current circumstance, but rather to encourage creative ideas for making the next 1-2 years into an enhancement to career.

Cohort Community. Belonging to a cohort going through a similar experience proved a powerful component of the prior CIFellows program. The current circumstances and their attendant challenges make cohort interactions all the more valuable, by providing peer support, enabling sharing of experiences and best practices, and creating a community of young researchers that can persist well past the current crisis. Further, we have the pool of prior CIFellows to tap for their input as we structure activities. These fellows are now 8-10 years into faculty or industrial research positions, ideally situated to reflect on what elements of the CIFellows program were most influential and to suggest new ideas to build the cohort and support the new Fellows.

We propose a set of cohort activities throughout the program beginning with a collaborative communication channel such as Slack. Once the academic year begins and Fellows are settled into their new role, we will begin a monthly seminar series drawing on content from CRA and CCC’s previous symposia and workshops around career growth.  For examples of these, please see CRA’s regular Career Mentoring Workshop agendas (CRA, 2019), CIFellows home page (CCC, 2009), 2014 cohort/career building workshop (CCC 2014), and 2018 capstone/next-stage-mentoring symposium (CCC, 2018). As part of our subaward monitoring, Fellows will be submitting quarterly reports detailing their research and related career enhancement activities. These will be shared across all of the Fellows to further additional career enhancement and cohort cohesiveness. We will also facilitate regular virtual meet-ups on topics of interest to the Fellows (both professional and personal).  We will ask for involvement from former CIFellows, where appropriate.  With involvement from current and past Fellows, we will organize a workshop on a regular basis, either virtual or in person, and continue it after the Fellowship has ended. 

3.2. Program Specifics

Eligibility. The Computing Innovation Fellows Program is open to all researchers whose work falls under the umbrella of the NSF Computing and Information Science and Engineering (CISE) Directorate and will have completed their PhD from a U.S. institution between January 1, 2019 and (expected) December 31, 2020. These dates are chosen to recognize that in some subfields of computer science (most notably, theory) it is common to do a postdoc before entering the academic job market. We wanted to make the program available to those with one year of postdoc experience, but not much more. The end date is chosen because some on the job market this spring may now delay graduation until the end of the fall term baring other options. Strong preference will be given to candidates who were on the academic computing research job market this spring and had their job options curtailed by COVID-19. 

Application process. Applications will consist of an academic CV, a letter of recommendation from the current advisor, one additional letter of recommendation, a career development plan, and a postdoctoral mentor letter. The career development plan will have two parts: a one page statement of goals for the postdoctoral experience, to include Intellectual Merit and Broader Impacts, and a one page plan specifying how the goals will be met over the requested timeline. The plan should address any anticipated issues that may require creativity in current times, such as remote mentorship and/or access to resources to conduct the proposed work. The proposed mentor letter must attest to a lack of funding for the recipient, commit to support the career development plan, and provide a mentoring plan. Guidance on mentoring plans will be provided as part of the call for applications (see Postdoctoral Mentoring Plan in Supplemental Documents). 

Selection Criteria (including COI rules for reviewing). A Selection Committee will be constituted to oversee the review process and make final recommendations for the CIFellow awards. PI Zegura and co-PIs Hill and Bradley will co-chair the Selection Committee, and 3-6 senior members from the CISE community will additionally be members of the Selection Committee. A Reviewer Pool will be made up of a larger number of senior researchers, based on subject matter expertise. The Selection Committee will establish the rating criteria for new CIFellows and communicate them to the Reviewer Pool. The Selection Committee will also be responsible for selecting and assigning reviewers to each application, managing conflicts, overseeing for consistency and quality of review, and facilitating discussion rounds. 

Evaluation will be holistic and will include metrics for diversity (of individual, research area, research methods, institution, geography, etc.). As in the prior program, the primary criteria for selection will be the strength of the applicant’s proposal and evidence of promise, with respect to intellectual merit and broader impact. Secondary criteria will be used to ensure a diverse set of students, research areas, and institutions. We will configure an existing application/review system familiar to CRA and the research community (e.g., HotCRP or Cadmium) to use in rating and selection. 

No one on the Selection Committee will be eligible to serve as a mentor or to have their students apply for the program. We will manage institutional and advisor conflicts in the reviewer pool using best practices such as not assigning applications to those with conflicts and removing those with conflicts from discussions of conflicted applications. 

Diversity. A central goal of the CRA is to foster a diverse computing research field, including individuals from different backgrounds, genders, institutional types, socioeconomic groups, ethnicities, etc. Applications demonstrating such diversity will be actively sought. In order to ensure unbiased selection and matching, the Selection Committee will undergo training to gain a greater appreciation for potential issues of applications from individuals of underrepresented communities (as was done in the prior program). Preference will be given to applicants who are U.S. Citizens or have Permanent Residency.      

Subaward Conditions. The subaward will be issued to the institution that will provide working space and basic resources for the postdoc. When the mentor and postdoc are at the same institution, this is straightforward, and the subward will be to the mentor. When the postdoc is supervised remotely, the subaward may be issued to the institution where the postdoc remains. In any case, the institution hosting the postdoc must provide suitable space and other basic resources to conduct research; specialized resources, and access to these if remote, must be provided by the mentor. As part of the application, Mentors will submit a letter of support, including a Mentoring Plan, as further described in the PostDoc Mentoring Plan Section of Supplementary Documents.  As part of the subaward reporting requirements, Fellows will need to report on the efficacy of the mentoring.

Discretionary Funds. As with any new position, there may be additional resources that are needed, such as travel funds, lab equipment, etc.  CRA will manage funds that Fellows can apply to use for in their fellowship to aid in their research independence and career growth. 

Dissemination and Launch. The CRA will use its well-established network to ensure widespread dissemination of the program details to the community. This will include the use of email, the Internet (including CRA and CCC social media channels, the CRA website and blog, the CCC website and blog, and the CRA government affairs blog), and the CRA Newsletter Computing Research News that is widely circulated across academic and industrial computing research organizations. As well we will post details on the CRA Job Announcements website which is the common site that is searched by those seeking employment in computing research. In addition, CRA will reach out to its member societies (ACM, IEEE Computer Society, USENIX, SIAM, AAAI)  as well as societies that work under the CISE umbrella such as ASA, ADSA, etc., for broad dissemination. 

We will advertise the program as soon as we receive permission to do so. We will have an aggressive timeline for applications and reviewing, expecting the process to take at most eight weeks from announcement to the community to notification of selection. 

4. Organizational Structure and Program Staffing

The CIFellows program will operate as a collaboration between the CRA Board of Directors and the CCC. We will establish a Steering Committee chaired by the PI of this proposal (Chair of the CRA Board) and including the co-PIs (Executive Director of the CRA, Chair of the CCC Council, Vice Chair of CCC Council, and Director of the CCC) and 1-2 additional people who have experience with the prior program. The Steering Committee will have oversight for the overall program, starting with the application and review process and continuing to cohort-building activities. The budget allows for the addition to the CRA staff of a person to handle logistics associated with the program, including managing the application and review processes, details of sub-award distribution, and coordination with other staff in CRA as needed.

The Computing Research Association (CRA) was founded in 1973 by early leaders of the field, based on a shared vision of community building and national needs. CRA is the most inclusive representative of the North American computing research community, with membership of over 225 academic institutions, 30 government and industrial laboratories, and the leading professional societies, including the ACM, the IEEE Computer Society, AAAI, SIAM, USENIX and CS-Can/INFO-Can, the Canadian computing organization. CRA is widely recognized by the United States computing research community as its representative organization. CRA helps advance the careers of researchers in academia and industry through its Career Mentoring Workshops and its Committee on Widening the Pipeline (CRA-WP). CRA works to increase diverse student participation in computing through CRA-WP and the Committee on Education (CRA-E). 

CRA has considerable experience in managing projects of benefit to the computing research community. Biennially, CRA brings together the computing research leadership at our Snowbird conference. When NSF recognized the need for a proxy organization to work with the computing research community to establish consensus and national research agendas, it chose CRA to create the organization—the Computing Community Consortium (CCC). 

Over the last five years, CRA has been awarded $10 million in funding for the CCC to catalyze the computing research community to enable the pursuit of innovative, high-impact research.  Many of the CCC activities are designed to create the researchers and leaders of tomorrow, on behalf of, and for, the computing community. With over thirty years of successfully meeting and exceeding its commitments, CRA is ready to undertake the proposed project.

5. Intellectual Merit

The principal way this proposal advances knowledge in CISE domains is through support of diverse research as proposed and conducted by all of the funded CIFellows. Based on the previous CIFellows cohorts’s performance, one can expect many advances from such a gifted, high-energy, and well-mentored group. Our review process will ensure that funded CIFellows have a track record of research success and a well-constructed plan to advance CISE research in interesting directions. The program also sets up a form of natural experiment about the impact of cohort programs and CIFellow designation on the future success of young researchers. Thus there will be an opportunity to conduct additional social science research to add to our understanding of what makes the most difference in launching academic careers. 

6. Broader Impacts

As with intellectual merit, the principal way this proposal provides benefits is through the union of the broader impacts created by all the funded CIFellows. One can expect many benefits from such a gifted, high-energy, and well-mentored group. The previous CIFellows have already had substantial impact, as discussed in the prior work section. 

In addition, this proposal has a critical direct broader impact: retaining a cohort of young gifted professionals in areas under the umbrella of NSF CISE to help create the future, as well as educate in areas critical to the 21st Century, thereby propagating the societal benefits further as the cohort subsequently advance in their careers. 

Having this talented cohort continue in research and teaching in CISE areas is critical due to trends in computing education. A recent article in “The Chronicle of Higher Education” (Kafka, 2020) makes a strong case due to the increased importance of computing education:

  • Four-fold increase in CS majors from 2006
  • The explosion of non-majors taking CS courses even at the uppers levels,
  • Many students are combining CS with other majors (“CS+X” or “X+CS”)
  • Faculty and teaching faculty positions have grown substantially but much slower than demand 
  • COVID-19 has underscored the importance of computing technologies, likely further increasing future CS interest

 The article concludes quoting Berkeley’s Dan Garcia, “We need everyone fluent in computing and technology, so that they’re not just passive users of it but active creators of it. Computing is the literacy of the 21st century.”

7. Results of Prior NSF Support

Andrew Bernat and Ann Schwartz Drobnis were co-PIs on the previous CIFellow awards.  (“The Third Computing Innovation Fellows Project,” #1136996, 8/1/11 – 8/31/18, $6,017,455, “The Second CI Fellows Project,” #1019343, 6/1/10 – 9/30/16, $14,995,174, and “Computing Innovation Fellows Project,” #0937060, 5/15/09 – 4/30/15, $14,988,394). Intellectual Merit: A robust research ecosystem depends on continuity of opportunity and experiences.  In addition to ensuring continuity through the CIFellows program, much was also learned about PostDoc experiences.  Those experiences have been captured and shared broadly with the community (through the Best Practices Memo and other CRA and CCC events), enhancing future experiences for all PostDocs in computing.  Broader Impacts: Maintaining researchers in the research pipeline and providing opportunities of growth for them not only enhances individual careers, but enhances the entire field of computing research.  Publications coming from these awards are Computing Community Consortium’s Computing Innovation Fellowship Program: 2014 Comparative Evaluation Report (Cundiff et al., 2014), Computer Science Postdocs – Best Practices (Jones & Gianchandani, 2012), and Lessons Learned from the CCC Postdoc Best Practices Program (Tamer & Wright 2018). These publications are used to create guidance for PostDocs in the computing community.    

Mark Hill, Liz Bradley, and Ann Schwartz Drobnis are co-PIs on the recent Computing Research Association award for the Computing Community Consortium (“Computing Community Consortium III,” #1734707, 4/1/18 – 3/31/22, $6,463,134).  The mission of the CCC is to catalyze the computing research community and enable the pursuit of innovative, high-impact research.  Two of the major goals of the CCC is to inculcate values of leadership and service and to inform and influence early career researchers to engage in community-led research challenges. Intellectual Merit: Through community visioning, the CCC creates future visions for the field, enabling new research challenges to be explored by diverse researchers across computing and adjacent research areas. Broader Impacts: By having a goal to engage early career researchers, the CCC helps to ensure diversity of thought in all activities, creating a richer ecosystem for the community.  The CCC has published countless workshop reports, presentations, and white papers for the community.  Recent reports are A 20-Year Community Roadmap for Artificial Intelligence in the US (Gil & Selman, 2019), Algorithmic and Economic Perspectives on Fairness (Parkes & Vohra, 2019), Content Generation for Workforce Training (Rushmeier, 2019). All reports were based on visioning workshops that included researchers from across computing, including early career researchers.  Recent white papers are 5G Security and Privacy – A Research Roadmap (Bertino et al., 2020) and Evolving Academia/Industry Relations in Computing Research (Patel et al., 2019). The Evolving Academia report, by Patel et al., has direct implications for the researchers who would be impacted by this CIFellow award. 

Ellen Zegura is co-PI on Institutional Transformation: The Role of Service Learning and Community Engagement on the Ethical Development of STEM Students and Campus Culture (NSF 1635554), $600,000, 09/01/2016-08/31/2021. Intellectual Merit: This proposed research aims to determine whether appropriately structured community engagement actively contributes to the moral maturation of students and facilitates the broadening of their sphere of ethical concern. Broader Impacts: The proposed research will inform the design of undergraduate STEM, and other educational efforts. To date, two female MS students and one male PhD student have contributed to the project. A new assessment instrument has been developed and released (Borenstein et al., 2019). In addition, the findings were published in, Undergraduate STEM Students and Community Engagement Activities: Initial Findings from an Assessment of Their Concern for Public Well-being (Erwin et al., 2018), and presented at the 2018 American Society for Engineering Education (ASEE) Annual Conference & Exposition. 


References Cited 

Bertino, E., Hussain, S. R., & Chowdhury, O. (2020, March). 5G Security and Privacy – A Research Roadmap. Computing Research Association (CRA). Retrieved by https://turing.cra.org/ccc/wp-content/uploads/sites/2/2020/03/5G-Security-and-Privacy-A-Research-Roadmap.pdf 

Binkley, C., & Amy, J. (2020, April 7). Financial Hits Pile up for Colleges as Some Fight to Survive. AP News. Retrieved from https://apnews.com/673bffcda00bf5522153c15e6e0373d5

Borenstein, J., Newstetter, W., Potts, C., & Zegura, E. (2019, May 2). Online Ethics Center for Engineering. Retrieved from https://www.onlineethics.org/Resources/GPRA.aspx 

Computing Community Consortium (CCC). (2009). CIFellows. Computing Research Association (CRA). Retrieved from, https://turing.cra.org/ccc/leadership-development/cifellows/

Computing Community Consortium (CCC). (2014). CI Fellows 2014 Workshop. Computing Research Association (CRA). Retrieved from https://turing.cra.org/ccc/events/ci-fellows-2014-workshop/#agenda

Computing Community Consortium (CCC). (2015). Postdoc Best Practices. Computing Research Association (CRA). Retrieved by https://turing.cra.org/ccc/leadership-development/postdocbp/

Computing Community Consortium (CCC). (2018). Early Career Researcher Symposium. Computing Research Association (CRA). Retrieved from https://turing.cra.org/ccc/events/early-career-researchers/#agenda

Computing Research Association (CRA). (2019). Career Mentoring Workshop. Retrieved by https://turing.cra.org/career-mentoring-workshop/

Cundiff, J. L., Wright, H., & Stout, J. G. (2014). Computing Community Consortium’s Computing Innovation Fellowship Program: 2014 Comparative Evaluation Report. Computing Research Association (CRA). Retrieved from https://turing.cra.org/cerp/wp-content/uploads/sites/4/2014/05/CI-Fellows-Evaluation-Report1.pdf

Erwin, A., Borenstein, J., Newstetter, W., Potts, C., & Zegura, E. (2018, June). Undergraduate STEM Students and Community Engagement Activities: Initial Findings from an Assessment of Their Concern for Public Well-Being. 2018 American Society for Engineering Education (ASEE) Annual Conference & Exposition. Retrieved from https://peer.asee.org/undergraduate-stem-students-and-community-engagement-activities-initial-findings-from-an-assessment-of-their-concern-for-public-well-being 

Fuhrmann, C. N., Hobin, J. A., Lindstaedt B., & Clifford P. S. (2011). Science Careers: Individual Development Plan. Science Careers. Retrieved by http://myidp.sciencecareers.org/

Gil, Y., & Selman, B. (2019, August). A 20-Year Community Roadmap for Artificial Intelligence Research in the US. Computing Research Association (CRA). Retrieved from https://arxiv.org/pdf/1908.02624.pdf 

Jones, A., & Gianchandani, E. (2012, December). Computer Science Postdocs – Best Practices. Computing Research Association (CRA). Retrieved from https://turing.cra.org/resources/best-practice-memos/computer-science-postdocs-best-practices/

Kafka, A. C. (2020, April 15). The Discipline That is Transforming Higher Ed. The Chronicle of Higher Education. Retrieved from https://www.chronicle.com/article/The-Discipline-That-Is/248536/

Parkes, D. C., & Vohra, R. V. (2019, May). Algorithmic and Economic Perspectives on Fairness. Computing Research Association (CRA). Retrieved from https://arxiv.org/pdf/1909.05282.pdf 

Patel S., Rexford J., Zorn B., & Morrisett G. (2019, June). Evolving Academia/Industry Relations in Computing Research. Computing Research Association (CRA). Retrieved by https://turing.cra.org/ccc/wp-content/uploads/sites/2/2019/06/Evolving-AcademiaIndustry-Relations-in-Computing-Research.pdf 

Rushmeier, H. (2019, March). Content Generation for Workforce Training. Computing Research Association (CRA). Retrieved from https://arxiv.org/pdf/1912.05606.pdf 

Singer, N. (2019, January 24). The Hard Part of Computer Science? Getting Into Class. The New York Times. Retrieved from https://www.nytimes.com/2019/01/24/technology/computer-science-courses-college.html

Tamer, N. B., & Wright, H. M. (2018). Lessons Learned from the CCC Postdoc Best Practices Program. Computing Research Association (CRA). Retrieved from https://turing.cra.org/ccc/wp-content/uploads/sites/2/2019/01/PostdocBP_FinalEvaluationReport_v2-1.pdf

            (Survey done end of March, 2020)

(Taulbee, set to come out May, 2020)



CRA has the facilities and resources to host the project: these being primarily web space within our cloud system. Additional facilities for each Fellow will be the responsibility of the host institution and will be spelled out in the appropriate subaward agreement.


Data Management Plan

Herein we present the proposal’s data management plan with each answer preceded in bold italics by the issues it addresses.

The types of data, samples, physical collections, software, curriculum materials, and other materials to be produced in the course of the project.

The data created will mostly include CI Fellows applications, letters of recommendation, peer reviews, evaluation discussion, evaluation decisions, and letters informing applicants.

This data will be processed via an evaluation management system to be determined. Leading candidates are (a) Cadmium in conjunction with CVENT that CRA uses for evaluating applicants for various opportunities and (b) HotCRP that is widely used by computer scientists to administer conference paper selection.

The standards to be used for data and metadata format and content (where existing standards are absent or deemed inadequate, this should be documented along with any proposed solutions or remedies).

Most data and metadata will be stored only in the evaluation systems selected. Items like letter informing applicants and reports including aggregated data may reside in CRA’s access-controlled storage.

The policies for access and sharing including provisions for appropriate protection of privacy, confidentiality, security, intellectual property, or other rights or requirements.

CI Fellows applications, letters of recommendation, peer reviews, evaluation discussion, evaluation decisions will be shared only with evaluators for the purpose of applicant evaluation. The letter informing an applicant will be shared with the applicant.

Aggregated (summary) data may be shared with the community for various reports and retrospective evaluations of the program.

The policies and provisions for re-use, re-distribution, and the production of derivatives.

There will be no re-use or re-distribution of raw data. The derivatives may use aggregated data.

The plans for archiving data, samples, and other research products, and for preservation of access to them.

All raw data will be preserved for three years (when most postdocs would be completed) and then destroyed to better guard privacy. We are open to other recommended preservation intervals.

The proposed project is unique, in that we will be creating a new postdoc program for Fellows in the CISE community, but the Fellows will not be mentored by people directly involved in the award creation / administration.  Each Fellow will have a mentor for their fellowship who will have their own style of mentoring, yet we want to ensure that a minimal set of best practices are followed.  To guide that process, we have created this postdoc mentoring plan for the CIFellows program.  This plan will be circulated to both mentors and Fellows as part of the application process.  We do not want to prescribe the individual mentoring plans, but we will ask mentors to include a formal mentoring plan, developed in conjunction with the Fellow, in their letters of support.  Review of these plans will be an important part of the review of the applications to the program.  Feedback from this review will be provided to the mentor for his or her use during the period of the fellowship.  Mentors of successful applicants will be asked to participate in a “Postdoc Best Practices” session provided by CRA in the summer of 2020.

Through the previous CI Fellows Program (CCC, 2009), PostDoc Best Practices Program (CCC, 2015), and the resulting evaluations (Tamer & Wright 2018), we have developed a good understanding of what works well for postdocs in computing.  It is critical that the CI Fellowship be a time for the postdoctoral researcher to gain independence and to have opportunities to lead in their research endeavors.  Based on these best practices, we encourage the Fellow and the mentor to consider the following strategies when developing the mentoring plan: 

  1. Fellow and mentor create a formal Individualized Development Plan (IDP) at the beginning of the Fellowship period (Fuhrmann et al., 2011).   
  2. Fellow and mentor meet regularly (at least quarterly) to discuss progress towards the IDP and make appropriate adjustments.
  3. Fellow has the opportunity to supervise or co-supervise other students.
  4. Fellow is guided to developing their own research agenda, including proposal writing.
  5. Fellow  is encouraged to network with new research communities, as defined by Fellow and mentor.  

In their second quarterly report to the CRA, Fellows will be asked to include a copy of the IDP and describe whether it is working out well, or if changes need to be made.

Permission to Share Proposal

To Whom It May Concern,

We are writing this letter as a supplement to the proposal titled Computing Innovation Fellows 2020 Project.  The proposal has been submitted to the National Science Foundation, under the CISE Directorate.  However, the Computing Research Association, Computing Community Consortium, submitting PI, and Co-PI’s grant permission to NSF to share this proposal with other potential funding agencies, both within and outside of the government.  If you have any questions, please do not hesitate to contact us.

Thank you,