Overarching Evaluation Questions
|
Methods and Metrics
|
Outcomes
|
Impact
|
- How is RMATRIX stimulating or generating translational research?
|
- Publications, presentations, patents, & grant proposals emerging investigators (pilot & junior) & RMATRIX researchers (KF members, mentors, HEALTH leaders in translational research)
|
Increase in translational research (from T0 to T4) over time
|
Effective, sustainable clinical and translational center in Hawai'i with significant community participation addressing health disparities among minority populations
|
- To what extent are potential and current investigators. needs being met by RMATRIX? How are new investigators navigating their way through the RMATRIX network?
|
- Satisfaction with services via interviews and surveys with emerging investigators & RMATRIX researchers
|
Improvement over time in the awareness, knowledge, usage, & satisfaction of RMATRIX services
|
- What new products, resources, knowledge, etc., has RMATRIX generated?
|
- Publications, presentations, grant proposals, & patents submitted & awarded by emerging investigators & key RMATRIX researchers
|
Increase in the number of publications, presentations, grant proposals & patents each year.
|
- How and to what extent has RMATRIX contributed to collaborations among researchers, institutions, and communities?
|
- Changes in collaborations with pre-/post- interviews with emerging investigators; review the number of co-authorships on grant proposals, publications, & presentations
- Assess frequency of collaborations among Key Functions using Social Network Analysis
- Assess collaborations among agencies, hospitals, community centers, & other institutions with pre- & post-test interview data from the Collaborations & Partnerships Key Function director
|
Increases in collaborations among researchers, Key Function areas, & institutions
|
- To what extent have health disparities in the community decreased because of RMATRIX?
|
- Pre- & post- interviews with emerging investigators to ascertain how their research has improved the health of targeted minorities in the community
- Track the number of studies by emerging & RMATRIX investigators focusing on targeted minorities over time
|
Increase in the number of studies focusing on targeted minority populations (Native Hawaiians & Pacific Islanders)
|
Specific Aims (related overarching quesitons in parenthesis)
|
Approach and Governance (1,2,4)
- Establish RMATRIX as lead C&T research infrastructure entity
- Engage C&T research development for researchers
- through MRET & CDP
- Establish key functions supporting & promoting multidisciplinary C&T research
|
- Establishment of RMATRIX; establishment of key function areas
- Project records on all advisory groups & consortia; number & types of collaborative arrangements
- Establishment of an investigator database
|
Infrastructure to support clinical and translational research in Hawaii
|
Collaborations and Partnerships (1,2,4,5)
- Develop collaborations & partnerships with university & community-based investigators and organizations with focus on reducing health disparities
|
- Increase in number of collaborative HEALTH projects
- Description of community members involved; assistance provided
- List of new researchers identified & developed with ORCHiD assistance
- Establishment & promulgation of community criteria for RMATRIX
|
Increase in local & university, community & regional partnerships
|
Community-Engaged Research (1,3,4,5)
- Establish ORCHiD and administrative structure
- Develop the infrastructure for implementing & disseminating a teaching program for academic researchers to learn about community-engaged research in Hawai'i
|
- Documentation that ORCHid activities, resources & products have been established
- Strategies to facilitate participation developed
- Community engagement criteria established, distributed & applied
- Documentation that linked investigator database has been created
- Criteria for CER established, distributed, applied
|
Greater involvement of community researchers in translational studies; increased involvement of academic researchers in CER
|
Multidisciplinary Research Education, Training and Career Development (2,3,4)
- Expand on existing masters and degree curricula
- Establish Community Based Research Training Advisory Program
- Provide career development activities
- Enhance collaboration with RTRN
|
- Documentation of ethics and biomed. Informatics integrated into the curricula
- Number & types of courses and workshops provided; enrollment figures
- Number and types of mentors ; development of mentorship program, including mentor training
- Number and types of career development activities
- Electronic surveys to measure program access & satisfaction
- Number and types of learning opportunities for community-based researchers
- Records of RTRN participation
|
Increase in graduates, investigators, and community researchers with translational expertise
|
Biomedical Informatics (1,2,3,4)
- Provide bioinformatics resources, tools and services
- Create a collaborative informatics environment that promotes translational & health disparities research and enables the networking of biomedical scientists
- Promote opportunities for investigators to enhance their knowledge and use of biomedical informatics; explore the roles of biomedical informatics on research
|
- Number of Hawaii and non-Hawaii-based collaborations facilitated by BMI
- Website with bioinformatics tools, services, data mgmt, transfer & storage capabilities; hit rate
- Number of partners mentored/ consulted on database, methods, & design
- Number of workshops/trainings supported; development of biomedical informatics course
- Number of grant proposals submitted; number of papers, posters, and abstracts submitted
|
Sustainable biomedical informatics infrastructure providing tools and services for translational research to reduce health disparities
|
Research Design and Biostatistics (1,2,3,4)
- Form a consortium of local programs, community partners, and national partners (e.g. RTRN, DTCC)
- Provide a centralized point of access to research design & biostatistics tools and services
- Ensure long-term sustainability of research capabilities by offering degree programs for investigators & introductory-level training for staff & community members
|
- Documentation of extent that core unit is developed, enhanced and utilized
- Baseline comparison & change in # of users, new queries, and/or users, tools/resource availability
- Documentation of tools and services offered
- Analysis of tools/service utilization and collaboration among partners
- Number of workshops & educational activities offered per year
- Number of students enrolled in courses and training
|
Coordinated and integrated biostatistics resources & capabilities to enhance collaborative, translational research
|
Participant & Clinical Resources(PCRs) (1,2,4,5)
- Restructure & transform existing CRC and CCRE to provide PCRs to support innovative research focused on HEALTH initiatives
- Facilitate participation in outpatient C&T research, especially by Hawaii.s underrepresented minorities
|
- PCR structure & operating processes governing the function of the 3 sites is communicated to RMATRIX leadership, KF leaders, UH, & the community; structure & operating processes posted on RMATRIX website
- SOPS in place for initial study review, PCR/study, PI function & responsibilities, & periodic monitoring
- Kakaako clinic renovations completed/Mobile van equipped & functional
- Increases in number of: PCR studies; pilot studies; studies in designated HEALTH initiatives; junior investigators supported;1st time PIs; grants submitted & awarded with intent to use PCR support; abstracts & publications crediting PCR; participants in PCR studies, especially Native Hawaiian & Pacific Islanders;
- User Satisfaction survey completed
|
Centralized infrastructure providing participant and clinical resources for translational research; Greater participation of minority populations in research, in particular, Native Hawaiians and Other Pacific Islanders
|
Regulatory Knowledge and Support (2)
- Centralize existing regulatory support services to serve RMATRIX investigators & interdisciplinary teams
|
- Documentation of progress has been made towards developing a website & centralizing services
- Documentation of user satisfaction with centralized services
- Reduced time to obtain compliance approvals
- Number of trained research personnel working with RMATRIX investigators
|
Reduction of regulatory barriers to translational studies; expedited research processes in translational studies
|
Ethics (2,4)
- Establish a research ethics panel to assist in addressing ethical issues
- Provide expert consultation & offer training
|
- Formation of ethics panel with relevant expertise
- Documentation of services; user satisfaction with services
- Number and type of training and consultation provided; number of participants
- Written procedures for identifying and addressing ethical issues in research
|
Integrated and expanded training and consultation in clinical research ethics
|
Evaluation
- Build a multidisciplinary evaluation team
- Conduct formative evaluation & communicate findings to leadership for program improvement
- Develop web-based data collection & tracking systems
- Conduct summative evaluations
|
- Document review; observations
- Review & document reports to leadership
- Examine electronic data collection & tracking mechanisms
- Examine survey & interview results; periodic & annual reports
|
Systematic, timely, and comprehensive assessment of RMATRIX; evaluation findings useful for program improvement
|