Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

17
Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success Robert M. Mayberry, M.P.H., Ph.D. Chairman and CEO χ 2 Health Research Consultants AcademyHealth AcademyHealth June 9, 2008 June 9, 2008

description

Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success. Robert M. Mayberry, M.P.H., Ph.D. Chairman and CEO χ 2 Health Research Consultants AcademyHealth June 9, 2008. What are the Objectives?. …improve access and quality of care. …improve health status. - PowerPoint PPT Presentation

Transcript of Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Page 1: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Community-based Organizations’ Capacity to Plan, Implement, and

Evaluate Success

Robert M. Mayberry, M.P.H., Ph.D.

Chairman and CEO

χ2 Health Research Consultants

AcademyHealthAcademyHealth

June 9, 2008June 9, 2008

Page 2: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 2

Page 3: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 3

What are the Objectives?What are the Objectives?

……improve access and quality of care.improve access and quality of care.

……improve health status.improve health status.

……prevent disease onset.prevent disease onset.

……insure fairness and justice.insure fairness and justice.

Page 4: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 4

Reducing the Racial/Ethnic Reducing the Racial/Ethnic Disparities: Framework?Disparities: Framework?

0

20

40

60

80

100

120

1 2 3 4

Time (decades)

Rat

e BlacksWhites

Page 5: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 5

Improving Quality, Reducing Improving Quality, Reducing DisparitiesDisparities

Sehgal JAMA 2003; 289(8): 996-1000.

Page 6: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 6

Hypothetical FrameworkHypothetical Framework

0

20

40

60

80

100

120

1 2 3 4

Time (decades)

Rat

e BlacksWhites

Page 7: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 7

Social Support Intervention to Social Support Intervention to Prevent Low Weight Birth amongPrevent Low Weight Birth among

African American Women African American Women

Fisher’s Exact Test (1-tail), P < 0.05

Norbeck JS et al. Soc. Sci. Med. 1996;43(6):947-954.

Birthweight Intervention Group Control Group

2500 grams or more 50 (90.9%) 45 (77.6%)

Less than 2500 grams 5 (9.1%) 13 (22.4%)

Page 8: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 8

A Three-Level Approach to A Three-Level Approach to Promoting Community HealthPromoting Community Health

Intervene intensely

when aproblem

is identified

Act Early

Build a wide family, community, and sociopolitical foundation

Page 9: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 9

Pfizer Southern HIV/AIDS Prevention Pfizer Southern HIV/AIDS Prevention Initiative Initiative

A 3-year initiative (2004 to 2006) designed to address the A 3-year initiative (2004 to 2006) designed to address the growing epidemic of HIV/AIDS in the Southern region of growing epidemic of HIV/AIDS in the Southern region of the U.S., primarily among racially/ethnically diverse the U.S., primarily among racially/ethnically diverse communities in urban and rural areas.communities in urban and rural areas.

Funds granted to small and mid-size community-based Funds granted to small and mid-size community-based organizations (CBOs) supporting underserved multi-cultural organizations (CBOs) supporting underserved multi-cultural communities in nine southern states: AL, FL, GA, LA, MS, communities in nine southern states: AL, FL, GA, LA, MS, NC, SC, TN, TX.NC, SC, TN, TX.

Partnership with Morehouse School of Medicine Prevention Partnership with Morehouse School of Medicine Prevention Research Center (MSM PRC) to strengthen the evaluation Research Center (MSM PRC) to strengthen the evaluation capacity of Initiative CBOs.capacity of Initiative CBOs.

Page 10: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 10

Characteristics of CBOs of the Pfizer Characteristics of CBOs of the Pfizer Foundation Southern HIV/AIDS Prevention Foundation Southern HIV/AIDS Prevention

InitiativeInitiative

Organizational Characteristic (Per Question Posed) Participant Responses (N = 39)

Yes

Do you have community volunteers? 89.7% (35)

Does your project have an established advisory committee? 59% (23)

Have you identified all appropriate resources needed to implement proposed activities? 61.5% (24)

Community Partnerships

Churches 53.8% (21)

Community-based Organizations 84.6% (33)

Colleges and Universities56.4% (22)

Local Clinics, Hospitals/Health Departments69.2% (27)

No Community Partnerships Exist 2.6% (1)

Page 11: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 11

Survey Responses of Program Staff Survey Responses of Program Staff Regarding the Technical Assistance Needs of Regarding the Technical Assistance Needs of

Community-Based OrganizationsCommunity-Based OrganizationsTechnical Assistance Needs Yes Reponses

(n=39)

Logic Model Development 48.7%

Data Management 48.7%

Data Collection Tool Development 61.5%

Protocol Development 53.8%

Qualitative/Quantitative Methods 64.1%

Evaluation Development 84.6%

Page 12: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 12

Cross-site Program Assessment Survey Cross-site Program Assessment Survey (C-PAS) Knowledge Level(C-PAS) Knowledge Level† † for Steps in for Steps in

Community Program DevelopmentCommunity Program Development

†Item mean score on a Likert scale of 0 (none) to 4 (extensive).

Self Assessment for Current Level of Knowledge for Steps in Community Program

Development

0

0.5

1

1.5

2

2.5

3

3.5

4

1st 2nd 3rd

C-PAS Administrations

Mean

Valu

e

Problem Identification

Needs assessment

Development ofgoals/objectives

Gathering programinput/feedback

Prevention interventiondevelopment/implementation

Evaluation ofcommunity intervention

Page 13: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 13

Cross-site Program Assessment Survey Cross-site Program Assessment Survey (C-PAS) Skills Level(C-PAS) Skills Level†† for Steps in Community for Steps in Community

Program DevelopmentProgram DevelopmentSelf Assessment for Current Level of Skill for Steps on

Community Porgram Development

0

0.5

1

1.5

2

2.5

3

3.5

1st 2nd 3rd

C-PAS Administration

Mean

Valu

e Problem Identification

Needs assessment

Development ofgoals/objectives

Gathering programinput/feedback

Prevention interventiondevelopment/implementation

Evaluation of communityintervention

†Item mean score on a Likert scale of 0 (none) to 4 (extensive).

Page 14: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 14

Cross-site Program Assessment Survey (C-Cross-site Program Assessment Survey (C-PAS) Summary Scores of Organizational PAS) Summary Scores of Organizational

Skills, Knowledge, and AbilitiesSkills, Knowledge, and AbilitiesC-PAS Administrations Skills†

Median (IQR)

Knowledge†

Median (IQR)

Abilities‡

Mean (SD)

1st (n = 39) 2.83 (0.67) 2.83 (1.33) 3.88 (0.42)

2nd (n = 36) 3.00 (0.75) 3.00 (0.66) 4.07 (0.45)

3rd (n = 41) 3.16 (0.83) 3.17 (0.34) 4.21 (0.46)

†Skills and knowledge based on a Likert scale of 0 (none) to 4 (extensive); summary score of six items. ‡Abilities based on Likert score of 1 (low) to 5 (high); summary score of 20 items.

IQR = interquartile range.SD = standard deviation.

Page 15: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 15

Pfizer InitiativePfizer InitiativePrivate-Academic-Community Private-Academic-Community

PartnershipPartnershipInitiative CBOs

•Community-based Programs and Partners•Innovative Approaches to Health Promotion

and Prevention

Pfizer Foundation

•Fiscal Resources

•Organizational Will

MSM PRC•Evaluation Expertise•Cultural

Competence

Page 16: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 16

Evaluation Capacity BuildingEvaluation Capacity BuildingRecommended ApproachesRecommended Approaches

Conduct Systematic Needs AssessmentConduct Systematic Needs Assessment

Adopt Practical Hands-on Learning Adopt Practical Hands-on Learning Opportunities Opportunities

Employ Quantitative and Qualitative MethodsEmploy Quantitative and Qualitative Methods

Facilitate On-Going TAFacilitate On-Going TA

Provide Opportunities to Share Evidence-based Provide Opportunities to Share Evidence-based PracticePractice

Page 17: Community-based Organizations’ Capacity to Plan, Implement, and Evaluate Success

Mayberry 17

Pfizer InitiativePfizer InitiativePrivate-Academic-Community Private-Academic-Community

PartnershipPartnership Critical partnership processes included:Critical partnership processes included:

– Cooperative Decision-making Cooperative Decision-making

– Systematic Communication StrategiesSystematic Communication Strategies

– Navigation of Unique Needs of Each Collaborative Navigation of Unique Needs of Each Collaborative Partner.Partner.