Table of Contents

1. Day 1

1.1. Welcome and Overview of Meeting

1.2. Public Participation NVPO perspective

1.3. Study Circles

1.4. AmericaSpeaks

1.5. Deliberative Democracy Consortium

1.6. Enhancing Public Participation through Information Technology

1.7. READII Project (CDC)

1.8. AAAS Public Engagement Project

1.9. Environmental Protection Agency (Office of Water)

1.10. National Aeronautical and Space Administration

2. Day 2

2.1. Welcome

2.2. National Institute of Allergy and Infectious Diseases/NIH, Division of AIDS

2.3. National Institutes of Health

2.4. FACA Considerations

2.5. Environmental Protection Agency (Superfund)

2.6. Department of Defense

2.7. Vaccine Policy Analysis Collaborative (VPACE)

2.8. Public Comment

Members in Attendance

Members

• Ruth Katz, George Washington University School of Public Health and Health Services-Chair
• Dr. Charles M. Helms, University of Iowa Hospital and Clinics
• Mary Beth Koslap-Petraco, Suffolk County Department of Health Services

Dr. David Johnson, Aventis Pasteur

Members Absent

• Dr. Jerome O. Klein, Boston University School of Medicine
• Dr. Patricia N. Whitley-Williams, Robert Wood Johnson Medical School

Dr. William Schaffner, Vanderbilt School of Medicine

Ex Officio Members

• Dr. Bruce Gellin, National Vaccine Program Office, DHHS
• Matt Leighninger, Study Circles
• Carolyn Lukensmeyer, AmericaSpeaks
• Tonya Gonzalez, Deliberative Democracy Consortium
• Robert Carlitz, Deliberative Democracy Consortium
• Ginger Pinholster, AAAS Public Engagement Project
• Victoria Friedensen, Office of Space Science, NASA
• Leanne Nurse, Office of Water, EPA
• Tamara Kicera, READII Project, CDC
• Rona Siskind, National Institute of Allergy and Infectious Diseases, NIH
• Jaynce Hedetniemi, Office of Community Liaison, NIH
• Doug Thompson, Superfund, EPA
• Jim Murphy, Superfund, EPA
• Ellie Tonkin, Superfund, EPA
• William Pehlivanian, U.S. Army
• Emily Marcus Levine, Office of the General Counsel, DHHS
• Mary Davis Hamlin, The Keystone Center

Invited Speakers

Audience members

• Deborah Wexler, Immunization Action Coalition
• Dr. Thomas Vernon, Chiron
• Dr. Thomas Zink, GlaxoSmithKline
• Dr. Geoffrey Evans, Health Resources and Services Administration, DHHS
• Dr. Kathleen Stratton, Institute of Medicine
• Jennifer Zavolinsky, Every Child By Two
• Sarah Landry, National Vaccine Program Office
• Maguerite Willner, Health Resources and Services Administration, DHHS
• Rosemary Gunn, Information Renaissance

Day 1

Welcome and Overview of Meeting

Ruth Katz

Ms. Katz introduced herself as a member of the National Vaccine Advisory Committee (NVAC) and Chair of the Public Participation Working Group. She noted that most of the working group members were present and thanked them for attending. She asked the working group members to introduce themselves and noted that two members, Dr. Whitley-Williams and Dr. Schaffner, were unable to attend. She thanked the presenters for coming and asked them and the public attendees to introduce themselves. She then thanked Ms. Landry for her work in putting this meeting together.

About a year ago, there was increased interest in enhancing public engagement in national vaccine policy. In June 2003, the Vaccine Policy Analysis Collaborative (VPACE) was brought to the attention of NVAC as a model for involving the public in setting national vaccine policy. VPACE served as a catalyst for the formation of this working group. The purpose of this group is to learn more about enhancing public participation, which will be reported back to the full NVAC at its October meeting.

At the conclusion of this two-day meeting, the working group will prepare a summary of this meeting and make recommendations on addressing this issue. Those recommendations will include a specific recommendation regarding the potential effectiveness of the VPACE proposal. The working group will also make recommendations about other potential approaches or models. The only issue before this working group is how to better engage the public, not whether to engage the public. There is consensus that there should be better public participation. How to achieve this is the issue at hand.

There will be 45 minutes allotted for each of the presentations and at the end of the presentations there will be time for public comments. Tomorrow morning, Dr. Helms will be chairing the meeting while Ms. Katz hosts another engagement.

Public Participation-NVPO perspective

Dr. Bruce G. Gellin

Dr. Gellin reinforced the idea that this is a question of how, not if, public engagement can be enhanced-the VPACE group also went through a similar process. The working group is also interested in hearing how other sectors have tried to solve this issue. The need for more public participation grew out of concerns with vaccine safety. However, there are a broad range of policy issues beyond safety, including pandemic influenza preparedness, production problems, resistance to an antiviral medicine, vaccine supply, and vaccine financing.

When discussing the pandemic influenza preparedness plan in DHHS, it became clear that this would be a broad-reaching problem. Should there be a pandemic, the implications for society are so broad that public engagement became essential. The draft plan has been posted on the website on August 26 for a 60-day public comment period. This meeting is, in part, an attempt enhance that public participation process as they move from the draft plan to the next stage.

Study Circles

Matt Leighninger

Mr. Leighninger noted that he began working with citizen involvement 11 years ago at the Study Circle Resource Center. The center was founded by Paul Edgar, who made a fortune in the electronics industry and went into philanthropy. Mr. Edgar started the center in a farmhouse next to his home. He was a Unitarian and joked that when Unitarians die, they have a choice between heaven and a discussion of heaven. The mission of the center was to get people in communities to talk about issues in small groups. Mr. Leighninger worked there for seven years and continues work with them as a senior associate. He also works with the League of Women Voters, Neighborhood Reinvestment Corporation, and the National League of Cities.

Over the last ten years, there has been a big shift away from traditional forms of citizen involvement. There are several reasons for this shift. First, particularly at the local level in the 80s and 90s, there was a mandate for dialogue among citizens on race-this was a clear issue for communities such as Los Angeles. Second, there was increasing acrimony surrounding policy decisions from the neighborhood level to the federal level. This, in part, might be due to the increasing ease of becoming an instant activist on any issue. Third, public resources were insufficient for solving public problems, such as education and crime prevention. Therefore, there was a need and desire to tap into citizen potential and organizations to work with government. Fourth, there were growing frustrations with traditional community organizing. In the past, community-organizing efforts were conflict based. Now, community organizers are using different tactics to engage the public.

Over the last 15 years, there has been a proliferation of citizen involvement or "democratic organizing"' efforts. These new efforts have the following key characteristics. 1) They are recruiting a diverse critical mass of residents. 2) They are using both small and large-group meetings. Small meetings provide intimacy and learning capacity, while the large meetings help them to see the potential of their efforts and have their conclusions amplified to decision makers. 3) They are giving people a chance to compare personal experiences and consider a range of views. 4) They are effecting change in learning, relationship-building, policy input, and tangible action.

There have been more than 250 communities, three states, and two national efforts that have conducted democratic organizing. Successful small groups (sometimes called study circles) share the following characteristics. 1) They include eight to twelve people and meet for several sessions-the more time they spend together, the better. 2) They are facilitated by someone who can remain impartial. This person does not have to be a trained professional, but must remain impartial. 3) They are open to many different views. 4) They include a diverse mix of people, including race, ethnicity, age, political affiliation, etc. 5) They follow a sequence from personal experience to larger questions to action planning.

Successful large-group meetings (sometimes called action forums) share the following characteristics. 1) They summarize the conclusions reached in the small groups. 2) They are used to launch task forces or committees on action ideas, or connect people with existing action opportunities. 3) They allow public officials to hear and add to the conclusions reached in the circles. 4) They provide participants a chance to socialize, network, and compare notes.

It is a misconception that these efforts are only about talking. There is a lot of discussion, but they are also about action at various levels. These include individual volunteerism, small-group action efforts, changes made by organizations, and policy changes. As an example of volunteerism, a participant in a Kansas City project to reconnect schools with their communities was quite skeptical of the process, but ended up volunteering to set up an after-school program around dance and the arts. For small-group action efforts, there was a project on police/community relations in Buffalo. One of the neighborhoods had an issue between halfway house residents and business owners. As a result, a mental health response team was established. For changes made by organizations, there was a discussion about waste disposal and landfills in a county in Kansas. As a result of that project, people concluded that recycling should become a bigger priority for the county. For policy changes, the League of Women Voters and others met to discuss the penal justice and correction system in Oklahoma. As a result, a state bill was passed almost unanimously, though similar bills were previously rejected along partisan lines.

There are several new assumptions about public involvement in these new efforts. One is that solving public problems requires the whole community. It is something that government cannot do alone. This results in a potential for greater problem-solving capacity. However, there is a greater need for coordination and at times confusion over accountability. It also illuminates the need for regional collaboration.

A second assumption is that proactive, network-based recruitment is critical. It is not enough to call a meeting and hope that people show up. By reaching out and being proactive, you can reach people who have not been involved. However, it raises the question of who should provide staff time for organizing. It reveals different outreach capacities of different groups and forces recruiters to think about "what's in it for citizens."

A third assumption is that putting policy decisions in citizen context helps the discussions. This creates a potential for more broadly supported policies, with people helping the government to implement the policy. However, this can be more difficult to address technical or abstract issues and requires different way of writing about issues.

A fourth assumption is that citizens can be expected to learn, empathize, and change opinions. When people are engaged, especially in small groups, they often do change their minds. However, there is a challenge of bringing together people on different sides of an issue and a danger for officials in suggesting that citizen opinions should change.

A final assumption is that there is a larger sense of civic duty, including dialogue, learning, and action. There is a potential for more people to take leadership roles. It requires a need to provide more opportunities for people to exercise active citizenship. However, there is a danger of division between "active citizens" and others.

There are several important planning questions when implementing democratic organizing efforts. These include:

• What is the issue you want citizens to address and how should you describe it?
• Who are you trying to involve and why would they want to participate?
• How will you provide participants with the information they need?
• Who are your key partners?
• How will you staff this project?
• How should you get started?
• What do you hope to achieve?

Regarding the work on vaccines, the best projects in public involvement include stakeholder collaboration, as well as public involvement. The first step in this type of project is building relationships with organizations that can help with recruitment. This relationship will also help show that the project is inclusive of all different or conflicting perspectives. This is exemplified in the VPACE proposal, which includes stakeholder participation and broader citizen involvement. Additionally, the issue of the flu is well suited for a national citizen involvement effort because it is something by which everyone is affected.

Questions

Ms. Koslap-Petraco asked about the mechanics of getting people involved who do not feel that their opinion counts. There is a need for broad spectrum of people. Those that are adamant about a particular issue are usually the ones that get involved. However, there is a need to hear from those that usually do not speak up.

Mr. Leighninger responded that the two key pieces are message and recruitment. The message must be clear that people will have the opportunity to make a difference and that the effort is not based on the agenda of only one group. On the recruitment side, it requires the work of reaching out to organizations to recruit within their memberships. Advertising is sometimes appropriate, but relationship building is by far the most effective technique.

Dr. Helms asked if it was critical to have agreement among stakeholders before the public is involved.

Mr. Leighninger noted that there does not have to be agreement on policy, only an agreement that this issue requires broader public input.

Ms. Landry asked about examples of when these efforts have not worked.

Mr. Leighninger responded that there are many examples. Many of these efforts fail because they do not have the right staffing, lack funding, or when facilitators cannot remain impartial.

Dr. Johnson asked about how these efforts could be translated to national policy or recommendations.

Mr. Leighninger noted that there are many options. One example is the efforts of the Canadian government. It involved a thousand small groups on immigration policies and was treated as an input gathering exercise. It provided materials and instructions to anyone who wanted to conduct a small group meeting, and the process was influential on policy change. Another idea is the concept of working with paired organizations across the country. There would be national organizations as part of a national coalition and coordinators who would work with different chapters. A third approach is to use two or three communities to get some representative policy input.

Ms. Wexler noted that stakeholder organizations tend to be those that have an issue or concern. She asked about how to engage the larger public that does not have a concern.

Mr. Leighninger responded that it would not be much different from the way citizens are engaged. You have to explain why the issue is critical and how the effort will allow their constituents to have some impact. In addition, there are a number of broad national organizations that have an interest in this kind of work and would be willing to participate.

Ms. Katz asked how the questions to be discussed are determined.

Mr. Leighninger answered that a committee of people with diverse viewpoints would frame the issue. After they figure out what the questions are, they would test the questions to make sure that they are not missing anything.

Ms. Katz responded that there is a separate group that looks at framework, rather than the group itself.

Mr. Leighninger added that his projects have all had the questions and issues set up before recruiting discussion group members. Without this level of specificity, it would be difficult to form a group.

Ms. Katz asked about the difference in small groups where the goal is to change minds versus the goal of building a consensus among those who will never change their opinion. Is this a matter of conflict resolution?

Mr. Leighninger noted that conflict resolution is different from citizen participation. There is some overlap, but conflict resolution requires more from the facilitator, who is skilled in that type of work. In most public participation processes, there is the hope that people from different viewpoints will come to some common ground.

Ms. Katz added that one of the keys to public participation is having people who are more open-minded.

AmericaSpeaks

Citizens Role in Policy Development

Dr. Carolyn Lukensmeyer

Dr. Lukensmeyer noted that one of the most challenging social issues of our time is the disconnect between decision-makers and citizens. This is especially critical in government since the foundation for American democracy was built on the idea of "of the people, by the people, for the people." The core question becomes how to put people back into their rightful role in a democracy in terms of influencing policy. The question of "how" to engage the public, rather than "whether" to engage the public, is a unique perspective. There are many in the country that still consider whether it should happen. All other countries plan and budget for public participation.

From 1987 to 1991, Dr. Lukensmeyer served as Chief of Staff to the governor of Ohio. The governor was concerned that they had passed many laws that match his platform, but they were unable to implement and change the bureaucracy. In that experience, she gained real insight into the frustration from the politician and citizen sides. Most of the rhetoric focuses on the gap between the government and the public in terms of a crisis of apathy-that citizens would not take the time and are not informed. However, the fundamental issue is that we have a crisis of alienation and disempowerment-people have real reasons for not voting.

In 1995, she spent six months traveling around the country listening to citizens who organized themselves to push initiatives that made a change. She discovered what motivated them, what they cared about, and how they engaged decision-makers. She learned two critical lessons. For the last two decades, the mantra has been that the government should be more like business. Though there have been benefits from this approach, this has left citizens feeling disempowered. A book published last year, Downsizing Democracy, talks about how public messages have reduced people's sense of agency. The second lesson was that citizens are extremely sophisticated about the role of special interest groups. There is a difference between the general interest public and the special interest public. In terms of recruitment, the formula is to get 60 percent general interest public and 40 percent representation of special interest.

The first time the 21^st Century Town Meeting model was used was on a dialogue on social security. Social security was a good example of how entrenched special interest groups were when it comes to privatization, benefit reductions, and tax policies. As a result, they created a dual track strategy. They worked with all of the stakeholder interest groups and created an advisory board made up of top leaders from various stakeholder groups. First, the board worked on advising on a participant discussion guide. This allowed all stakeholders to put forth their positions. After months of debate, all members of the board approved the guide and called it fair and equitable.

Second, the public felt closed out of the process by experts. When they would attend public hearings, they often felt demeaned, disrespected, and made to look foolish by what they did not know. In Western culture, experts are not expected to translate their knowledge for the general public. As a result, public processes must be designed to include expert knowledge in a way that does not disempower the public. It must also allow the public to understand that good public policy requires the collective knowledge and wisdom of everyday people.

For example, Ohio's governor tried to radically reform education. To do this, he never met with experts first. Instead, he met with those who were most impacted. The first meetings were in eight cities in Ohio with populations more than 100,000. There he listened to kids in grade school, elementary school, and high school, and their parents. From these meetings, he learned that the most significant issue was safety. In 1985, safety would not have been on the agenda of any education expert.

For decision-makers, there is an issue of timing, which has been exaggerated since the 1980s. For example, politicians were expected to have a full policy position on terrorism immediately after the World Trade Centers fell. It is absurd where we have taken the role of media to demand politicians immediately to take a position on issues. Conversely, corporate executives are rewarded for changing their business strategies. Yet, politicians are rejected for changing their positions.

The mission of AmericaSpeaks is to engage citizens in the most important decisions that affect their lives. This process has been used at the national level with social security and a new project on youth obesity is about to begin. They have also used this process with U.S. mayors in setting budget priorities in their communities. They have done regional planning and helped in the decision-making process in the rebuilding of the World Trade Center site.

She presented a picture of the Jacob Jarvis Center in New York City, the site of the World Trade Center town meetings. These meetings had to be large enough to have the credibility that it actually represents all of the constituents of New York. In the picture, there are 500 round tables of ten to twelve people, where intimacy is gained at the table itself. Each table was diverse, in terms of gender, race, age, and class.

As a side note, she added that the economy of all major cities is dependent on undocumented workers. The low-end jobs at the World Trade Center were almost entirely held by undocumented workers-many of them died. AmericaSpeaks joined with an existing effort to give employers amnesty so that these workers could be identified. This allowed the meetings to have credibility with organizations that worked with these undocumented workers. As a result, 448 of the 5,000 meeting participants were undocumented workers.

For a successful meeting, it must be an informed dialogue. For the World Trade Center meetings, there were maps, conceptual designs of the architecture, and budget spreadsheets. Additionally, there is a tremendous amount of human support in the process. For example, when people came to discuss social security, they stated that what they really cared about were constituency issues. As a result, they had actual social security counselors on site to help the participants resolve their problems.

To go from intimate table discussions to a collective view of thousands of people simultaneously, they used standard information technology that has been used in private companies for large business meetings. Every table had a wireless laptop computer for citizens to collect input. The discussions were about 45 minutes, with 20 minutes being the shortest and one hour as the longest. One of the core topics dealt with was who the memorial was for-victims, families, New York, world, etc.

While the tables are collecting data on their laptops, those data go in real-time to a bank of computers managed by Theme Teams. The data are then placed into categories by themes, which also include minority reports. Once the data are collected into themes, they are projected onto large screens throughout the hall. Each theme included various amounts of options (data). These options were then voted on and prioritized using polling keypads, which were provided to each participant. The keypads were numbered and tied to the demographics of the voter.

It was important to the Port Authority to know how family members felt, so they agreed that family members would be over represented. Each table had one family member of a victim. So that that person did not have to carry the emotional burden of the table, there were at least two other persons who were hugely impacted by 9/11-emergency workers, people who lived near the site, etc.

After each day, preliminary reports were copied on site and given to each participant (decision-makers, media, and citizens). The public read and discussed these reports enthusiastically and with a high sense of empowerment. They took the reports to their churches, businesses, or rotary clubs to share the information. These reports also increased the sense of agency in the citizens.

There are four critical success factors. First, every voice must be at the table-all must be represented. Second, decision-makers must be involved at each step and must influence (not control) the process from the very beginning. If decision-makers only see the finished results, they have no choice but to treat it like a special interest group. Third, the content, structure, and process must be correct. Fourth, there must be outcomes that make a difference.

There were five fundamental lessons learned.

Scale transforms the potential for impact and enables people to experience the community as a whole. Small discussion groups are important, but they do not show the experience of the whole community.

Through active and creative recruitment, it is possible to reflect the full diversity of the community. The myth of "hard-to-reach" groups is untrue and results from using the wrong medium. Posting notices and radio ads are not always the correct communication vehicles. Outreach strategies must be specifically targeted to every relevant demographic.

Transparency is critical to people's sense of ownership of and confidence in outcomes.

Citizens are capable of dealing with complex information and making trade-off decisions.

The quality of citizen recommended outcomes are only as good as the quality of the design of the materials and the process.

Questions

Mr. Johnson noted that the examples seem to be one-time/one-issue processes for input. He asked whether those that require a process over a long period of time were doable under this mechanism.

Ms. Lukensmeyer replied affirmatively. None of their projects were done just once. Social security is an example of a long-term process. It was done over 18 months. Additionally, the mayor of Washington, D.C. has been using this process to set the city's budget since 1999.

Ms. Wexler asked how much time was involved for the participants and how they were able to get off work. She also asked who pays for travel expenses.

Ms. Lukensmeyer noted that, as a principle, no one is paid to participate. She added that events are primarily held on Saturdays or Sundays, while accommodations can be made based on demographics. They did pay for transportation, fed people well, and listened to what the community said they needed to participate.

Ms. Koslap-Petraco asked how the money was raised.

Ms. Lukensmeyer noted that each project was a mix of government, foundation, and private funds. There was one exception, social security. The Pew Charitable Trust wanted to be a source of nonbiased, nonpartisan information about social security, so they paid for the entire project. She added that the government in Canada and certain European countries have a budget for public consultation.

Dr. Helms asked who the decision-makers are in national immunization issues-Congress, the public health service, etc.

Ms. Lukensmeyer noted that if decisions could be made by the agencies, it could be treated administratively. If legislation is involved, as in social security, both houses of Congress and the White House needed to be involved.

Ms. Katz noted that in their social security project, Congress ended up doing nothing. In light of the public's cynicism, would they be even more cynical since nothing happened.

Ms. Lukensmeyer added that they did not have the funds to track those people. However, she did not feel that citizens necessarily become more cynical. During the social security meetings, they allowed materials and people in the lobby from all advocacy groups. This allowed participants to know and communicate with the various advocacy groups. For example, a participant, who changed his views to favor some government control over privatization, linked up with the Kato Institute to try to influence their actions. Additionally, the project website continues to get 20,000 visitors that keep the discussion current. If you link people to something at the local level that allows them to continue forward, it is possible that the experience did not raise people's cynicism.

Ms. Katz asked if the complexity of the issues (e.g., transportation versus vaccines) matters to the success of these projects.

Ms. Lukensmeyer offered an example. In Denmark, the Danish Technology Board is responsible for doing a number of public engagement processes on a range of issues. One of these engagements dealt with biogenetic engineering. At these meetings, the outcomes of what the citizens say move directly to the debate in Parliament. In summary, the public can be educated on anything.

Ms. Katz noted that Oregon took the issue of Medicaid to the voters directly. She asked whether this same process should be used for vaccines.

Ms. Lukensmeyer noted that she is a proponent of representative democracy. She added that referendums have been distorted by big money. For example, she noted that each presidential election cycle has its top issue. In 1992, healthcare was the top issue and there was a mandate by the Americans for a national solution. This issue could have been taken out of the partisan legislative process and put into public dialogue. The results could have been moved directly to the House floor, bypassing the committee structure.

Dr. Zink asked what the costs are.

Ms. Lukensmeyer replied that it ranges from $70 to $400 per person (New York).

Mr. Johnson noted that there are a number of fundamental questions in immunizations. There are also questions on individual vaccines and vaccine interventions. This process can be used for larger questions, but could the process be applied to short-term issues where public input is needed very rapidly?

Ms. Lukensmeyer noted that the Department of Agriculture has a service called the USDA Extension Service, which has two-way satellite communication capabilities in every state. This is already funded by the U.S. government and could allow for a rapid turnaround. There are two issues that would most determine the length of the process. One is the development of materials. The second issue is outreach-getting people in the room. Though it drives up costs, an alternative is using a scientific random sample.

Ms. Wexler noted that undocumented workers live in crisis and that it is difficult for them to prioritize a discussion on vaccines. She asked how you could get people who do not have time in the first place to get involved in an issue that may not be very important to them. She noted that many people do not take the time to even vote.

Ms. Lukensmeyer invited Ms. Wexler to see the process and noted that this was the best way to understand how it is done. In response to the idea that people do not vote, she added that she learned that people do not vote because they do not believe that it is a fair system. She added that people would be more inclined to discuss vaccines than to vote. Once they go through several of these discussions where they see how they have made a difference, then they might begin to feel that the system is more fair and vote.

Deliberative Democracy Consortium

Tonya Gonzalez

Ms. Gonzalez introduced herself as the Director of the Deliberative Democracy Consortium (DDC), which was started two years ago by AmericaSpeaks. The idea of DDC from the beginning was that practitioners and researchers in the field of deliberative democracy thought they could do more together than they could alone. Ms. Lukensmeyer is the chair of their steering committee, which is comprised of representatives from most of the major public participation organizations, both in the United States and abroad.

The nascent movement for deliberative democracy in the United States and around the world served as the impetus for starting DDC. Deliberative democracy, which is about engaging citizens and giving them a voice in decision-making, is the cornerstone of our democracy. Outside of the United States, deliberative democracy has a long history-from the Danish Board of Technology to, more recently, the World Social Forum in Brazil.

DDC is led by a steering committee, which addresses critical areas of work through task groups-National Infrastructure, Knowledge Building, Link to Government, and Online Deliberation. Dr. Carlitz is a member of the Online Deliberation Task Group. These task groups work on questions, such as:

• What methods work best when?
• How do you carry a one-time event over a long process?
• How do you create a more deliberative public?
• In what are we engaging people?
• How do you educate people about complex issues and is there a dialogue that needs to happen first?
• Social movements in this country have been determined by advocates' views, so where does this work fall into deliberative democracy?
• If you have a group of parents or a community that thinks something else should be done about vaccine policy, how do you get them to come to the larger consensus?
• What are the costs involved?
• What are the carry-on effects?
• What mix of methods should be looked at?

Questions

Ms. Katz inquired whether a group should go to DDC to ask about the best method for a particular issue, rather than just go directly to an organization such as AmericaSpeaks.

Ms. Gonzalez responded that a group with such questions could go directly to AmericaSpeaks, but DDC would be a resource that offered a different level of understanding.

Dr. Zink asked who could join DDC.

Ms. Gonzalez responded that anyone could join. They have members from CDC and EPA. Other members include Senators, researchers and practitioners of the field, people who come from related practices (e.g., conflict resolution), and those who have participated in some type of deliberative forum.

Enhancing Public Participation through Information Technology

Dr. Robert Carlitz

Dr. Carlitz stated that he is part of a non-profit organization called Information Renaissance (IR). IR was founded in 1996 and grew out of several infrastructure projects to connect schools, libraries, and community centers to the Internet. It evolved to using that online infrastructure as a mechanism to facilitate citizen participation in government. IR views itself as the intersection of public need, governmental resources, and the technology tools that can merge these resources together. Rule making is a primary interest of IR, for it has enormous potential for citizen involvement. The notice and comment process invites input from many interested parties, which is strongly weighted by the agencies as they make their rules.

Traditionally, it was difficult for citizens to participate in rule making because a paper-based system required the participants to be physically located where the information was. However, the process is moving online, with 24-hour access. This shift has the potential of affecting a major change in how things work. Currently, online rule making is largely implemented out of the CIO office of most agencies

The potential reach of online events is enormous. Online events are simultaneously nationwide and do not require a series of hearings around the country. Additionally, online mechanisms allow for comprehensive resources to be built overtime, and materials from hearings and research can all be stored in one place.

A key issue in public participation is how managing information and authoritative sources. Authoritative records, which are meant to be accessible by the public, should be updated in an online repository. This achieves a high degree of transparency because the materials are in their entirety, rather than edited down versions, such as press releases.

Online resources are available on demand from anywhere in the country-participants do not have to travel to any particular site. Additionally, the online environment is also a good place for background and educational materials. As a result, it is possible to construct an online resource around a particular issue, which not only shows the proceedings of a particular committee, but also provides background materials. The net effect of online processes is a deepening of the level of public participation. In addition to people being more aware, they participate in a more intelligent and more informed manner.

Online techniques can be applied anytime the process involves public input, such as proposed legislation, planning for local construction projects, issues scoping within an agency, proceedings of Federal Advisory Committees, rule making, and the implementation specific legislation.

The preparations for an online event are very similar to an in-person event. People need to be notified and recruited to achieve balance in terms of demographics. In the participatory planning phase, the agenda is developed and the panelists and participants are determined. As people sign up for the event, the online mechanism can capture biographical information. There can be surveys to map out the territory and get a summary of attitudes, which can inform modifications to the agenda and set or adjust priorities.

Online mechanisms are a combination of the informal and anonymous. People who are normally afraid of speaking at a public meeting would be willing to speak in this more anonymous online forum. Additionally, agency members who appear remote and inaccessible in hearings can be made more personable.

Online events allow for participant introductions. They can provide biographies and collect demographics. They also allow for participant feedback. Basic resources can also be available, such as surveys and a comprehensive library so that participants have access to a complete set of information. Through this, it should be possible for citizens to have the same information that the experts do to analyze the issue. Additionally, online events allow for participant feedback, which can help set priorities and agendas.

Online events are self-documenting. Materials put online stay online, unless an effort is made to remove them. As materials accumulate, they can be indexed and searched. This creates an important archival component that allows non-participants to learn about the process.

Deeper online participation can help deepen public participation. "Websites" or "online resources" become collaborative workspaces. People actively participate in the process and are not just reading what is there. They provide information and input. They can determine or influence the agenda and help set priorities, which is accomplished through a voting mechanism. These processes get people more interested in the activity and allow them to feel that they have a voice. There can be on-going monitoring of results either by the group as a whole or various subgroups.

Another technique is notification. A problem with websites is that they can rapidly accumulated a large amount of material, making it difficult for participants to keep up with the information. A solution is a technology called RSS feeds, used by newspaper sites to collect headlines of articles. RSS feeds can be sorted by keywords, resulting in a customized newspaper with new items in various areas of interest show up.

Online dialogue involves pulling together a group to talk online about a particular subject. These events typically involve 1,000 people and have lasted two or three weeks in length, which has proven to be a convenient time period. They tend to be intense and involved, and have generated an enormous amount of interest. When staff support has been available to digest the data from these events, it has been extremely useful for the agencies involved. Online dialogues involve the following elements:

• Moderator-to help organize the discussion
• Threads-organizing the discussion around particular topics
• Discussion summaries-daily summaries posted back to the group
• Tracking-notification if someone responds to a posting
• Archive-information resource
• Panelists-to launch particular topic areas

There are problems with online events. 1) People without network access cannot participate. Possible solutions to access include public access sites, community sites, libraries, and government offices. There is also the evolution of technology in telephone access, interactive television, and other mechanisms becoming more broadly available. Though these solutions are creating greater access, online access is presently a real problem. 2) Reaching certain populations that are not showing up at these events require particular effort. 3) Literacy is a problem universally for public participation. If people are not articulate, afraid to stand up in front of a group, or lack the vocabulary to address the issues at hand, it is going to be hard to get them to participate. 4) There is also the issue of comfort levels with technology.

There are also benefits. 1) Young people tend to be more comfortable with computers than standing up and giving a speech at a public hearing. 2) Rural populations, young parents, and people unable to travel will be able to participate online. 3) People who cannot participate during the workday will also be able to join the online discussion because it is available 24 hours a day. Most of what is done online is asynchronous, meaning immediate participation is not necessary-a message is posted and a reply can be posted later. 4) People with disabilities who are comfortable with computers have easy access to this type of forum. 5) The background materials and library of resources available online allow interested participants, who do not know a lot about the subject, to learn more and become informed participants.

There are several concerns with online events. 1) The sponsoring group needs resources and expertise to host an online event. Merely announcing an online event is not enough, often resulting in off-topic comments or no comments. Online events require more preparation than a public hearing because they are nationwide. 2) Legal constraints to online events are beginning to be considered and are showing up in law journals and literature. These constraints are problematic because they are new and often interpreted differently from one agency to another. These interpretations are mostly based on comfort level with the technology. On one side, online events are just a different vehicle for public participation. On the other, these events are revolutionary and should be approach with caution. 3) In collecting data, there are restrictions in terms of privacy issues and paperwork reduction issues. The EPA's general counsel, for example, believes that it is an invasion of people's privacy if their names shows up in an author index if they have submitted a comment online. A search capability allowed contributions to be found, but author indices were suppressed. Another question involves whether the actions of a moderator trying to keep the discussion on topic are violating free speech rights of the participant. Similarly, restrictions on prolific commenters is problematic.

There are advantages to face-to-face events. They can be more personal and allow for facial and body language to be seen. However, online events allow for more detailed information and structure. Face-to-face and online events can also work together. Online mechanisms can be used to announce face-to-face meetings, which can be tracked online. For example, face-to-face meetings can be broadcast over the web at low costs, called webcasts. This can include video clips and archives of previous meetings.

Generally, online discussions can be more open-ended than face-to-face meetings because they can stretch over a longer period of time. As a result, online resources can serve as a comprehensive, long-term coordinating structure for all public participation events and all of the information associated with these events.

Questions

Ms. Katz asked if Dr. Carlitz had been involved in any work where the same question was looked at online and in face-to-face meetings. She asked if the results were compared for differences in the response.

Dr. Carlitz responded that one of the early events was done with Ms. Lukensmeyer on social security reform. There had been a number of face-to-face meetings, accompanied by an online discussion. The online discussion tended to be more in-depth and covered a longer period of time. Congressional staffers who participated were interviewed after the event for their opinions of the process. They noted that in their day-to-day jobs, they tended to produce sound bites for their bosses. However, they found members of the public to be interested in more detail and enjoyed sharing their research and expertise by answering questions in detail. Some staffers put an enormous effort into the event.

Ms. Katz inquired whether the issues brought up online and at the in-person meeting were similar or different.

Dr. Carlitz replied that they were similar, partly due to the fact that each group was exposed to similar background information.

Dr. Johnson posed the question of how people are recruited to participate online and if they self-select.

Dr. Carlitz responded that they are self-selected. They tend to advertise through groups that already have a presence online. They post announcements and ask people who are interested to tell their friends about it. If the groups are well organized, the announcement will be success. In the social security example, the online audience was skewed toward Libertarians because they were very active in talking about this topic online, and they were excited about talking to people who were writing legislation on this topic. A flaw in online mechanisms is that the demographics of the participants lean towards the demographics of the online community. Surveys on recruiting mechanisms show that online mechanisms worked best.

Ms. Landry asked if there is a criteria on selecting online library materials?

Dr. Carlitz explained that they try to work with bi-partisan forums in selecting the materials. In the social security discussion, there was a White House conference on social security that had bi-partisan sponsorship. The groups who were invited to participate in that meeting were asked to submit a one or two-page statement of their outlook. Their statements, showing all sides of the various issues, became key materials in the online library. Groups that were not involved in the White House conference were also allowed to post their statements. It is important not to be perceived as being biased towards a particular position.

Ms. Landry inquired if there were criticism from people who thought they were not giving enough opportunities.

Dr. Carlitz responded that during the social security event, someone posted a message stating that the event was a way for Prudential to push its agenda on privatizing social security since part of the funding came from Prudential. There was a response by a Libertarian who said that the participants were being fooled into thinking that the forum was actually open and inclusive. There was criticism, but in general, the participants believed in the process.

Ms. Wexler posed the next question on whether there is usually a specific issue asked to be addressed online?

Dr. Carlitz responded that with the FCC rulemaking event on connecting schools and libraries to the Internet, there was a preamble that posed a number of questions. They structured the events so that each topic had equal time, and the discussions were given specific topics to cover-it is ideal to have a focus and an outcome. The social security discussion was weak because it did not end with a policy change. Rulemakings are nice, because after the 60-day period is up, a rule is issued. People will participate when they know there will be an action.

Ms. Koslap-Petraco asked how participants are recruited. She works with an underserved population. She cannot imagine her Hispanic patients getting involved in something online, which would result in an enormous segment of the population being left unheard.

Dr. Carlitz agreed.

Ms. Koslap-Petraco asked if this would be an adjunct to face-to-face meetings.

Dr. Carlitz noted that online events would be an adjunct to face-to-face meetings. One thing that motivates people to participate online is the opportunity for their voices to be heard worldwide. Audiences that are too dispersed or cannot access the online medium can participate through intermediaries, using small in-person meetings.

Ms. Koslap-Petraco wanted more information on how to get the word out to isolated and dispersed populations.

Dr. Carlitz noted that online events use the same techniques as other events. However, the most successful are online advertisements and mechanisms. It is also important to reach out to groups that represent those that are isolated and dispersed. He added that they have done a lot of work in low-income areas, which has a lot of interest. Opportunities to participate can be provided at library and community sites.

Ms. Gonzalez commented that complex issues require longer citizen engagement, rather than single events. The social security event had mixed components in the discussion, and the best approach may be a combination of methods to get the level of recruitment for which you are interested. There is a learning curve and as online events go on and people begin to realize that they do want to have input, recruitment may look very different two years from now.

Ms. Koslap-Petraco suggested doing some of the recruitment in languages other than English.

Ms. Gonzalez added that AmericaSpeaks and Study Circles do ongoing work with various populations, and materials for Study Circles for various events have been translated into Spanish.

Ms. Koslap-Petraco commented that she did not want to see Hispanics left behind.

Dr. Carlitz responded that even with the expense of translation, it has to be done.

READII Project (CDC)

Tamara J. Kicera

Ms. Kicera noted that READII stands for Racial and Ethnic Adult Disparities in Immunization Initiative. She noted that the READII demonstration project is somewhat different from the earlier presentations with respect to soliciting public input-there is a difference between getting assistance with implementation and policy or decision-making. For the most part, READII projects engage partners to implement intervention strategies that have already been shown to be effective.

Ms. Kicera presented an overview of the READII project. She indicated that she would not be presenting much data or background information during this presentation since she is scheduled to do so at the October NVAC meeting. Instead, she would be focusing on experiences with the Rochester or Monroe County, New York project. She noted that Dr. Nana Bennett and Christine Long, the leads on this project, were unavailable to attend this meeting.

One of the key messages from the Rochester experience was that starting or funding something new is not always necessary. Existing community groups, stakeholders, or infrastructure could perhaps be enhanced or expanded rather than starting from scratch.

The READII project genesis began in Spring 2002 when Dr. Walter Orenstein (NIP/CDC director at the time) made a presentation in the DHHS Secretary's office about the need to take a closer look at the racial and ethnic disparities among older adults, particularly with respect to the influenza and pneumococcal vaccinations. DHHS was supportive of this initiative since eliminating racial and ethnic disparities has, and continues to be, a top priority. On July 31, 2002, DHHS announced that the project would move forward. They immediately moved to engage other federal agency partners, including CMS, HRSA, AoA, and AHRQ. They then awarded the funds to the five demonstration sites late that year. After obligating the funds, they quickly moved to identify stakeholders and other community partners, and develop community plans to begin the demonstration projects.

The immunization programs being funded had little or no adult infrastructure in place, unlike the longstanding, well-supported childhood immunization programs. Unfortunately, many immunization and health department personnel were asked to respond to the smallpox vaccination programs being implemented at the same time, and with the influenza season that had already begun, a tremendous burden was placed on these personnel.

Ms. Kicera then discussed what is currently happening with the project. She noted that sites and agencies formed new, important partnerships and have actively engaged their communities. They are better understanding infrastructure gaps and the difference between childhood and adult programs.

The five READII sites are Chicago, Milwaukee, San Antonio, Rochester, and the Delta Region (19 counties in Mississippi). The sites have been encouraged to employ evidence-based interventions, most of which are taken from the Community Guide for Preventive Services. Some of the more effective ways for increasing adult immunization levels include standing orders and provider recommendations. Formative research is generating ideas for targeted health education messages to patients and providers, including small-scale media campaigns.

Ms. Kicera showed a slide of Dr. Julie Gerberding (CDC Director) holding one of the provider tool kits that was developed as part of the READII project. The tool kit was a set of materials intended to help providers understand the importance of their vaccine recommendations, particularly with respect to elderly African-Americans and Hispanics. She added that the READII project activities have been widely publicized, particularly in READII project site communities.

Ms. Kicera discussed what has been learned. First, new challenges are faced every year. Challenges in the first year included smallpox efforts, SARS, and monkey pox. Another challenge was the early flu season last year, in which there were highly publicized pediatric deaths resulting from a vaccine shortage. She noted that duplicating childhood successes for adults may not work and that new methods, interventions, and partnerships might be necessary. She added that they are trying to form partnerships with agencies such as AoA since many of the existing partnerships target preventive services for children rather than adults.

Ms. Kicera noted that negative attitudes among health care providers, especially nursing staff, can negatively affect the success of even a standing orders policy. In Rochester and Chicago, many private practices do not have the traditional structure of providers with many registered nurses (RNs). Because of costs, nursing shortages, and other reasons, many practices operate without any RNs. Instead, they operate with nurse's aides and LPNs, who do not necessarily have the education level or attitudes about the importance of promoting vaccinations. Even in places with standing orders, staff with doubts or misconceptions about vaccines move away from recommending vaccination to high-risk patients.

Ms. Kicera emphasized that changes of this magnitude take time and they have not done very well with the general public with respect to influenza and pneumococcal vaccinations levels and are doing worse with minority populations. To effect such change cannot be done in a two-year project, but would require eight to ten years of targeted intervention to see a dramatic change.

A mid-course review meeting took place in March 2004 with the five grantees, the federal partner agencies, and several senior officials from the Deputy Secretary's office and other DHHS agencies. The purpose of the meeting was to review and discuss successes and challenges thus far in the project; to allow discussion of issues and challenges with CDC, partners, and invited panelists; and to share site-specific project information with senior officials from DHHS and other partners.

The demonstration projects are scheduled to end December 31, 2004, followed by aggressive evaluation activities. The experiences with the 2003-04 influenza season present opportunities. In response to the media frenzy about being immunized, facilities in Rochester were flooded by white, middle-class, or suburban populations, while minority groups did not have the same reaction. Determining interventions that will be most effective with black and Hispanic elderly populations is a challenge. CDC has been charged with creating an influenza vaccine stockpile and hopefully the demand will be there. Experiences in the five sites will be valuable to future programmatic and research efforts. Some NVPO funds have already been used to launch studies about attitudes, beliefs, and misconceptions of nurses and other healthcare professionals with respect to vaccinations. The findings will be disseminated with the hope that successful efforts will be replicated.

Ms. Kicera then discussed Rochester's Community Action Plan using slides presented by Christine Long in Atlanta. This was a collaboration of the University of Rochester and the Monroe County Department of Public Health. The Community Action Plan was developed at the front end of the project and included the following components: a community advisory board; community-based organizations; a communications plan; several targeted projects; and eight or nine private vaccine provider groups, which have shown very promising results. The primary care practice intervention had other components, including a communications strategy, providing public vaccine, targeted projects, and community-based organizations.

The Community Advisory Board engaged groups such as the Urban League of Rochester, economic development groups, senior citizen organizations, and a coalition of African-American churches. The people in Rochester felt it was critical to address these groups at the front end of the project. She noted the importance of involving decision-making groups at the front end, rather than developing a report to present to them at the back end of the project.

Ms. Kicera noted that Rochester, Milwaukee, and Mississippi targeted black, elderly populations. San Antonio targeted the Hispanic elderly, and Chicago targeted both blacks and Hispanics.

In Rochester, the goal of the Community Action Plan was to promote the messages that vaccines are important and necessary, and that disparities do exist in the community. Another goal was to provide direction on where and how to get vaccines. She also noted that none of the five sites were overtly trying to solicit input from communities about policy or the recommendations or interventions being promoted. The recommendations and interventions came from the Community Guide for Preventive Medicine or ACIP. The demonstration sites were asked to engage their community stakeholders to help promote these messages.

The groups did not want new projects funded or new groups brought in because they felt there was a wealth of resources within the community that could be worked with or expanded upon. They found that one-to-one or small group formats worked best and that there was a need for patience because these types of changes take time. Most groups that were members of the Community Advisory Board volunteered their organizations through in-kind contributions. They volunteered to be speakers with the media or church groups.

The Community-Based Organizations (CBOs) were, for the most part, engaged in outreach and education for their own constituents (e.g., foster grandparents, case management clients, Head Start programs). The group from the health department working specifically on the READII project supplemented these activities by providing low literacy brochures and training for the CBO staff. Feedback from the CBO staff indicated that the training was extremely helpful in terms of raising their awareness about the importance of the vaccines.

In the first year, there were 23 staff members from 12 CBOs that participated in outreach training on adult vaccines and cultural competency, and ten CBOs routinely reported on the activities (e.g., how many people they were reaching, number of events they sponsored, any forum where immunization activities were being promoted).

While Rochester targeted their African-American populations, they found that they needed to do a better job of engaging their Hispanic community. There was not only a need, but also an interest among the Hispanic groups to participate. They also wanted to engage CBOs from other parts of the city. Rochester found that one problem was identifying providers that serve minority, elderly populations. There was initially the assumption that providers serving minority pediatric populations would either serve the adults or direct them to adult providers. This was not the case and identifying providers in the community who saw a large number of the African-American elderly was difficult.

Ms. Kicera then discussed some of the communications. The NIP/CDC conducted focus groups at each of the five sites. One of the messages was not to limit the discussion of immunizations to the influenza or the influenza season. For example, pneumococcal vaccine can be given anytime during the year, but is often coupled with influenza vaccination campaigns. In Rochester, they found that priming people to consider talking to their providers in the summer about being immunized in the fall was very helpful in terms of moving people towards getting vaccinated. Another message that resonated in the fall was that being vaccinated will also protect loved ones.

Ms. Kicera discussed the lessons learned from the Community Action Plan. First, the interpersonal approach worked best-working with CBOs, churches, senior housing, etc. They found that African Americans in Rochester did not respond to the national media frenzy that created last season's influenza vaccine emergency the same way that white populations did. It is difficult to evaluate whether it would be more beneficial to invest funds in media campaigns versus more targeted activities with community groups or providers.

They emphasized that the involvement of the Community Advisory Board contributed to the success of the interventions. Incorporating the message into their ongoing activities is better than beginning a new campaign. Trusted members of the community promoted these messages.

Questions

Dr. Johnson asked about taking national policy recommendations to the community and whether there was any sense of pushback from the community on these policies.

Ms. Kicera responded that there was a lot of pushback and commented that they were surprised that Rochester was one of the sites with so much resistance. Rochester has a long-standing relationship with CDC with respect to immunization policies. She noted that the demonstration projects that took place 10 to 12 years ago, which resulted in CMS placing influenza vaccine as a covered benefit, occurred in Rochester. The provider community in Rochester was already primed about the importance of providing immunizations, so the resistance they encountered from providers was surprising. Some providers refused to implement standing order policies. In facilities where standing orders were implemented, many nurses and other staff often were from the same minority groups as the target patient populations and shared many of the same myths and misconceptions about the vaccine. As a result, if patients asked these nurses whether they should receive the flu shot, the nurse would advise against it. Rather than nurses advocating for immunization, there was a lot of subtle, negative information passed around. She emphasized the need for an effort to understand these negative attitudes, to educate these workers, and to increase immunization rates among healthcare professionals.

Ms. Katz asked whether there was any reason to believe that resistance was because the people were not involved in the formulation of the policies, or whether there would have been resistance regardless of their level of participation.

Ms. Kicera responded that they were hoping to do additional research in this area to get a better understanding. She noted that feedback showed that some providers, through liability issues with implementing standing orders, did not like relinquishing control over their patients. Although standing orders are routine in many settings, they encountered the same resistance in hospitals, nursing homes, etc. She added that it appeared that providers were putting up barriers that are not true barriers. In facilities where standing orders have been implemented, there have been dramatic increases in immunization rates without any liability issues with respect to screening and immunizing patients meeting specific criteria. She noted that in larger practices or managed care facilities, if one can approach the person who makes policy decisions (e.g., medical director, administrative manager), the policies work well and the providers are happy that this does not take their time away from their patients. However, in smaller practices, providers are stretched in terms of the time they have with their patients and the number of messages they are expected to relate. Adults do not visit their providers for preventative medicine-they go in when they are ill or have some type of chronic medical condition requiring follow-up visits. Even though Medicare reimburses for the cost and administration of the vaccine, providers do not make money and in some cases lose money, particularly in small settings where overhead is high, time is short, and billing Medicare is complicated. Some providers, while they recognize that immunizations are important, particularly for high-risk patients, hope that patients get immunized elsewhere.

Dr. Zink asked about alternate provider sites and whether any of the demonstration sites tried to collaborate with different delivery systems.

Ms. Kicera noted that this has been done, but less so in Rochester than the other sites. She noted that CDC is recognizing that they will have to move away from the childhood model to succeed with adults, and that they will need to effectively partner with mass immunizers and other ways to immunize large numbers of adults. As seen with pandemic influenza planning and the smallpox experience, infrastructure for immunizing large masses of adults is not established in this country. If we had to gear up quickly to do this, it would be very difficult. They have looked at working with, for example, pharmacies. However, not every state allows pharmacists to immunize. State law and legislative differences influence these efforts. In Rochester, other than physicians, only RNs can administer vaccines, and as discussed earlier, many facilities do not have RNs on staff.

Dr. Zink asked whether nurses who had negative opinions about vaccines had views that varied from one year to another depending on the specific situations-whether some years are good matches as opposed to others.

Ms. Kicera responded that this was not the case. They found that myths and misconceptions of healthcare professionals were echoed by the patient population (e.g., the flu vaccine gives you the flu). She noted that they received "schizophrenic" messages during work with some of the focus groups, where, for example, there was distrust of government healthcare programs, particularly among African Americans. However, their most trusted messengers were their healthcare providers. They noted that if their healthcare providers recommended it, they would get it. In most cases where adults have not been immunized, it was because their providers did not tell them they needed it. She commented that the attitudes seen among healthcare workers have not been sophisticated in terms of good matches one year versus another year.

Ms. Koslap-Petraco commented that she was trying to fit this into public participation in terms of the healthcare provider group, particularly nurses since she is one of them. She noted that healthcare providers need to be educated first. This was the topic of a paper she wrote for CDC. In her health department, immunization rates went up after an educational program was provided for healthcare workers. Workers were asked their concerns about why they did not want to be vaccinated and the educational program was built around these concerns. She clarified that in the state of New York, RNs are the only nurses permitted to vaccinate, but LPNs can vaccinate under the supervision of an RN. She noted that in Suffolk County, pharmacies hire RNs to administer vaccines. She acknowledged that this is more complicated on a national level with different laws in different states.

Ms. Kicera added that one thing they found surprising going into providers' offices was the level of disorganization and state of disarray. In many cases, the providers could not identify their high-risk patients or even patients 65 years of age or older. The providers indicated that they did not have the time or resources to do this type of organization, but told READII staff that they were welcome to come in and do it for them. Ms. Kicera noted that they had been under the impression that the providers could just pull up a list of, for example, all patients over 65. This was not the case.

Ms. Nurse asked how much of the attitudes found among minority groups were related to issues of aging (i.e., these people are old and going to die anyway and do not need special treatment).

Ms. Kicera responded that this has been a topic of conversation between Roger Bernier (NIP), herself, and others for a number of years. She noted that they are baffled by the lack of outrage at the staggering number of hospitalizations and deaths from influenza and pneumonia in this country-90 to 95 percent of which are occurring among older Americans. If these types of numbers were to occur among any other population group, there would be a tremendous public outcry about what is being done to prevent it. The vaccine recommendation is for people 50 and over, many of who are still very active. An estimated 114K to 115K excess hospitalizations and 50K to 60K deaths occur annually from vaccine-preventable illnesses. They are having difficulty understanding the societal attitudes around the aging issues. As the population continues to age, they hope this issue becomes a national priority.

Ms. Nurse asked about the use of media research in terms of the efficacy of print campaigns versus radio and third party campaigns. There is increasing evidence that people are not trusting print media campaigns, particularly those from government sources. Because of the oral tradition in black and Hispanic communities, people will listen to their neighbors, ministers, etc. She asked whether this recent research is being utilized in terms of radio and third party campaigns.

Ms. Kicera responded that Mississippi and Milwaukee have launched aggressive radio campaigns. They have also looked at alternate sites such as church-based groups. Mississippi recently had a huge faith-based summit in which Garth Graham from the DHHS Office of Minority Health participated. She noted that disparities between the African-American and Hispanic elderly and the general population persist despite controlling for things such as socio-economic status, access to healthcare, insurance coverage, etc. They have therefore, in addition to implementing interventions that have been shown to work, encouraged creative, new ways to reach out to these communities.

Leanne Nurse invited Ms. Kicera to present the Rochester findings at the 2005 Community Involvement Conference in Buffalo.

AAAS Public Engagement Project

Ginger Pinholster

The American Association for the Advancement of Science (AAAS) mission is to "advance science and innovation throughout the world, for the benefit of all people." The practical goals are to preserve the long-term AAAS reputation (circa 1848), increase visibility, and engage the public. She noted that in her office, the Office of Public Programs (OPP), the specific mandate is to rapidly, equitably, and efficiently communicate with the public and reporters worldwide. This is relatively new-they used to be the news and information office at AAAS.

OPP uses conventional press techniques to reach out to the public. They have also used animal wrangling, such as Rico the Wonder Dog, a dog that can remember names of things through a process of "fast-mapping" or cognitive elimination. Like most science authors, he initially refused to talk to the press, but unlike most science authors, he crawled under the table to get away from Reuters. Traditional approaches are press briefings, followed by public briefings.

In 2004, AAAS had two initial public engagement activities. Family Science Days was held at the AAAS annual meeting in Seattle and drew 10,000 people. Oceans for Everyone town hall meeting was another activity.

A key public engagement vehicle for all AAAS work in reaching out to the press and the public is EurekAlert!, which is their editorially independent website that reaches 540,000 public visitors per month. Research news is presented in a variety of languages and there are 4,800 reporters from 50 countries registered with this site. One of the most important destination-specific sites on EurekAlert! is the Science for Kids feature. This was started as a way to keep the science press package team interested in their jobs. The site now draws some 10,000 public and reporter visitors per month. A spin-off of this feature is the "What Would Kids Ask?" public engagement effort. Each week, two children are invited to engage directly with Science authors by asking them questions. This idea came from a public briefing with the three cloned mules where a child asked Dr. Gordon Woods whether the mules had the same memories. Dr. Woods responded that they have different birthdays.

Ms. Pinholster then gave a brief background on the public appetite for science based on information AAAS disseminated to the public and the press. She showed a list of the best selling Science features and noted that they have not included much for children in terms of medical or health news.

She noted that at each AAAS annual meeting they serve about 1,200 press registrants. She also highlighted the stories that generated the most coverage in 2004, with stem cell research as the top story. Based on the latest Science press package survey, medicine was at the top of the list of what individual reporters wanted to know about from Science and AAAS. She noted that the more whimsical topics (e.g., dinosaurs, animals) were towards the bottom of this list, while 2003 coverage trends showed that animals were popular. This is a result of the difference between what reporters want and what their editors may be able to get in newspapers or what producers may be able to put on television. She commented that it is encouraging that the topic of health threats is among the top three categories and noted that there is tremendous coverage whenever there is a health-related paper.

Ms. Pinholster then discussed the AAAS Center for Public Engagement with Science and Technology. The two primary goals of this new center are to provide an open forum for the public's concerns and hopes regarding scientific advances, and to provide an opportunity for the public to help frame future research agendas. The mandate was set forth by AAAS CEO, Dr. Alan Leshner, in Science magazine. This is a departure from their well-respected, 30-year old public understanding initiative, which is more education-driven. Dr. Leshner had noted that it may not be realistic to assume that the public can always understand science and technology issues in the same way that a physician or scientist might, but this does not mean that the issues do not affect their lives or that they have no right to have any say.

Ms. Pinholster then discussed the Oceans for Everyone town hall meeting. The meeting was initially a half hour briefing for 60 reporters with five speakers. It grew and was advertised as a three-hour event. It lasted four hours and there were over 230 attendees, including 34 local policymakers, and five speakers. Ira Flatow was recruited as the moderator.

The town hall meeting was structured as two simultaneous events. There were a series of breakout or dialogue groups, followed by the central town hall meeting that was kicked off by Ira Flatow and Alan Leshner. At designated intervals, the breakout groups would have their discussions and then feed summary reports to Ira Flatow. Their input was summarized to rank the choices they felt would be the most realistic approach to deal with ocean issues in the Seattle area. Results were posted on the web and passed out to participants as they left.

They worked with one of their media partners, the Science Channel, for a real-time webcast played at the Annual Meeting Internet Café, a delayed broadcast radio spot, and an advance feature piece for television. They also worked with Public Agenda, a public engagement group, and Porter-Novelli's social marketing unit to put this together.

Porter-Novelli conducted a public opinion survey of 2,400 adults in advance of the event to get data to use as a news hook for a premise for discussion. Adults know that the health of the ocean is being endangered and yet, very few feel empowered enough to do anything. This led to the second, indirect message that AAAS is there to help.

Ms. Pinholster commented that they were impressed with the work of Public Agenda, but that perhaps Porter-Novelli's help was not necessary. In terms of participation, they expected about 300 participants and ended up with 230. One lesson learned was that they should over book. The twelve collaborating organizations were not pushed very hard to work through their communications channels to recruit people. This affected the socio-economic and ethnic diversity of the group and some groups were under-represented. She commented that the breakout groups did not work as well as planned. The choice work model given by Public Agenda was overly complicated for what AAAS was trying to do. She did not feel they needed that level of programmatic complexity. Their biggest problem was that some of the panelist discussions were too long-winded. There was, however, good dialogue. Dr. Leshner gave this first effort a B+.

The proposed format for future town hall meetings is to use a reactive-mode format that may stimulate dialogue and insights. As stated earlier, all AAAS public engagement activities are intrinsically linked to their press outreach activities. Much of their success depends on how well they convey their message to the media. She gave the example of The Washington Post article on the 2003 event at MIT where the Dalai Lama and Eric Lander were both asked to look at the color blue and explain what went through their minds. With this model, the event becomes the news. It is also possible to harness the power of celebrity to reach many more people.

She added that they are interested in public health as it affects families and children. She also displayed a list of other possible topics that they have under evaluation.

Questions

Ms. Katz asked if, from the AAAS point of view, the purpose of the town hall meetings was to get media attention to the issue.

Ms. Pinholster responded that this was not the only purpose, but was critical to their success.

Ms. Katz asked how the topic of oceans was chosen.

Ms. Pinholster responded that the topic was chosen primarily because of the venue. They knew there was a strong marine science community in Seattle. They looked at the peer-reviewed programmatic content for the meeting and saw that a significant percentage had to do with marine science. They spoke extensively with a number of partnering organizations that were willing to work with them. They felt that it would work and that there would be public demand for this type of information.

Ms. Katz pointed out that this was different from the previous presentations in that this was more of an educational event-AAAS did not have to make a decision about what had to be done with the oceans.

Ms. Pinholster noted that AAAS was not as interested in engaging the public in policymaking as they were with raising awareness about some of the issues and giving the public the opportunity to participate.

Ms. Wexler asked if this was their first foray into getting the public engaged.

Ms. Pinholster responded that this was the first town hall that they had done.

Dr. Zink noted that there was interest in the United Kingdom about engaging the public and scientists, particularly with respect to genetic foods. He asked for comment on any AAAS involvement or study in this area.

Ms. Pinholster responded that they have had no involvement. She commented, however, that this was an interesting question because they saw the 9/9 UK announcement about the availability of public engagement money, and with the major nanotechnology seminar coming up at the 2005 AAAS annual meeting, one can imagine this being video-conferenced with the United Kingdom.

Ms. Landry asked if there was any feedback from the participants on whether they felt this was worthwhile, whether there has been any measurable change, or whether more people are now interested in the issues.

Ms. Pinholster responded that they conducted a post-participation survey and people pretty much graded it the same as Dr. Leshner (B+). People commented that some of the speakers were too long-winded and there was some grandstanding that the moderator was not able to contain. The questions that were never fully answered were what to do next and how to sustain the interest. They established a marine science portal on EurekAlert!, which has become a very viable virtual community for many of the participants.

Dr. Vernon raised the issue of stem cell research, an area where people are likely to have more hardened positions. He noted that this is more the type of environment they are likely to encounter with immunization policy discussions. He asked Ms. Pinholster if she could speculate about what might be done differently under these conditions.

Ms. Pinholster responded that, as Dr. Helms had commented earlier, everyone invited must agree on the premise that they will not necessarily agree on the specific points, but agree that they are worth discussing. With Public Agenda, the level of advance staging was phenomenal and they anticipated virtually every comment and prepared a moderator for responding to them.

Ms. Wexler asked about the 2,400 people surveyed before the meeting and how they were selected. She also asked whether the 230 participants were drawn from this pool.

Ms. Pinholster responded that the number was from Porter-Novelli's random sampling. She added that the 230 participants did not necessarily come from the 2,400 surveyed-they worked with the 12 collaborating organizations to recruit participants.

Ms. Katz asked whether the audience had any difficulty understanding the science.

Ms. Pinholster responded that she believes anything can be explained, and added that they did not include a high level of technical detail.

Environmental Protection Agency (Office of Water)

Common Sense to Common Practice-Improving Public Involvement at EPA

Leanne Nurse

Ms. Nurse's presentation focused on work in the recent Executive Order on Cooperative Conservation that the White House issued last week, as well as new EPA Administrator Mike Leavitt's initiative on collaborative problem solving. She noted that there is some controversy over cooperative conservation. Some people feel it is a timely, election-oriented effort to show how friendly the administration is and how much it is listening through the agencies.

However, this kind of work has been happening at EPA for some time, in terms of social science and environmental research. The largest body of information on civic engagement is in the environmental field, much of which is a result of EPA's legislative mandate. They have a lot of good work on which they are building. Their office, the National Center for Environmental Innovation, was tasked with developing the policy for public involvement. Bob Carlitz from Information Renaissance will help her discuss with more detail on the online dialogue that helped them formulate and launch the policy.

She expressed her appreciation for Patricia Bonner and noted that she is very ill. Ms. Bonner has been a pioneer at EPA and many know that she developed its customer service policy, which provided leadership for the federal government. Similarly, the public involvement policy is the first agency-wide policy. They are getting a lot of feedback from their federal partners through ongoing discussions, which NVAC will hear about tomorrow with the Deliberative Democracy Consortium and Link to Government Group, and through high-level conversations at the Council on Environmental Quality (CEQ). The goal of this work is to enable common sense things that they agree on (e.g., being respectful in listening and giving timely feedback) to become common practice across an agency with 18,000 people.

EPA has a huge job in explaining and building the body of knowledge showing that effective civic engagement and public involvement creates better environmental outcomes. This is a hard sell to business and science-oriented managers who have been convinced through their training and social models that they are the experts, they have the answers, and if we trust them with the work, then the outcomes will be good. That is meeting the rising public expectation that ordinary people have informal knowledge and have other kinds of expectations and information that need to be taken into account as public decisions are made.

They have, through a long series of conversations with citizens and experts in the field, developed the "Seven Basic Steps for Effective Public Involvement." She noted that copies of the presentation, a brochure that answers basic questions, and online tools are available.

The most important of these steps is the one least likely to take place, which is to plan and budget for public involvement activities. Most people have been involved in situations where someone calls a meeting quickly to check it off their list and then calls it public involvement. Fortunately, there is a lot of good work throughout public and private organizations where people recognize that it is helpful to include stakeholders in the strategic business model. For many, this idea continues to be a surprise and an affront to their boundary issues. It is important not to take for granted that many people do not understand the value of building relationships as part of their business model, their pathway to effectiveness, and their final decision-making. Care must be taking in how managers and technical people are addressed.

Ms. Nurse then reviewed a chart outlining the spectrum of decision-making and public participation processes. This chart has been particularly helpful for managers as they walk them through the EPA collaboration process. She then reviewed the International Association for Public Participation (IAP2) public participation spectrum and the range of activities. They can talk about collaboration and can have negotiated settlements, but all of this more complicated work is built on very basic steps, such as providing basic, effective, and timely information. The meeting attendees may be familiar with the IAP2 spectrum, which can be found on the IAP2 website.

She then reviewed the consultative processes EPA uses-how they actually conduct business across the spectrum of engagement. They do basic things like tours, workshops, and roundtables, which are essentially one-way means to get out information. Unfortunately, this is the bulk of what happens, but they are also aware that through advisory committees, the FACA process, and joint task forces with various partners, they are able to have a more productive and thorough ongoing dialogue. The collaboration work that the Administrator and the White House have been discussing is to look at some these larger scale conversations and situations in which an opportunity exists to equitable, negotiated settlements. Power sharing is among the issues people are facing. How do those who have power share that power? How do those who have access to proprietary information agree to share that information with those they do not trust upfront?

One of the tools that they are using and that may be useful for NVAC to examine is a contract through their Center for the Prevention and Resolution of Conflict. It is an alternative dispute resolution (ADR) contract (www.epa.gov/adr), which allows third party neutrals to work for the agency and to be involved in situations where the agency many not be the convener of record, but may be a participant in a particular situation.

The tools they have include a series of booklets, which the Working Group will receive. They have brochures that include the Seven Steps and discuss working with the environmental justice community and improving public meetings and hearings. A lot of this material came from the dialogues that Information Renaissance helped them develop, the public comment process, and the areas of ongoing learning, such as appreciative inquiry. They are now looking at appreciative inquiry specifically for internal marketing and to help develop a community of practice in the agency.

They have a small network called the Public Involvement Improvement Council, and representatives from each of the program offices-such as Air, Water, Solid Waste Emergency Response, each of the ten regional offices, the laboratories, and the Office of Research and Development-are involved. They are planning a two-day retreat in November in which they will use a process of strength-based or appreciative inquiry. Looking at good practices, they will look at where they are now, how to get where they want to be, and how to build on what they already know.

They also have laminated cards and bookmarks, and a portable exhibit that they are revising for different settings. They are also waiting for the next fiscal year to release money for training, which will provide some of the baseline work that is being done in the agency to improve the practice of high-level practitioners. This will begin to provide some leveling of the knowledge for others in the agency about basic practices, as well as a set of five modules for managers that are less about classroom training and more about engaging them in teachable moments. A key issue is how to reach managers when they are already "flooded out" in their respective meetings or situations and how to tailor materials to address specific program issues and managers' responsibilities. At the Senior Executive Service (SES) meeting, with which Tonya Gonzalez helped, the managers were adamant that the agency respect an