Included are important news articles from various sources that pertain to education today. Occassionally there are a few tips and tricks relating to education throughout the blog.

Saturday, June 14, 2008

Amazing Teacher Facts

From: The Wall Street Journal
http://online.wsj.com/article/SB121339775502373623.html?mod=opinion_main_review_and_outlooks

This month 3,700 recent college grads will begin Teach for America's five-week boot camp, before heading off for two-year stints at the nation's worst public schools. These young men and women were chosen from almost 25,000 applicants, hailing from our most selective colleges. Eleven per cent of Yale's senior class, 9% of Harvard's and 10% of Georgetown's applied for a job whose salary ranges from $25,000 (in rural South Dakota) to $44,000 (in New York City).

Hang on a second.

Unions keep saying the best people won't go into teaching unless we pay them what doctors and lawyers and CEOs make. Not only are Teach for America salaries significantly lower than what J.P. Morgan might offer, but these individuals go to some very rough classrooms. What's going on?

It seems that Teach for America offers smart young people something even better than money – the chance to avoid the vast education bureaucracy. Participants need only pass academic muster and attend the summer training before entering a classroom. If they took the traditional route into teaching, they would have to endure years of "education" courses to be certified.

The American Federation of Teachers commonly derides Teach for America as a "band-aid." One of its arguments is that the program only lasts two years, barely enough time, they say, to get a handle on managing a classroom. However, it turns out that two-thirds of its grads stay in the education field, sometimes as teachers, but also as principals or policy makers.

More importantly, it doesn't matter that they are only in the classroom a short time, at least according to a recent Urban Institute study. Here's the gist: "On average, high school students taught by TFA corps members performed significantly better on state-required end-of-course exams, especially in math and science, than peers taught by far more experienced instructors. The TFA teachers' effect on student achievement in core classroom subjects was nearly three times the effect of teachers with three or more years of experience."

Jane Hannaway, one of the study's co-authors, says Teach for America participants may be more motivated than their traditional teacher peers. Second, they may receive better support during their experience. But, above all, Teach for America volunteers tend to have much better academic qualifications. They come from more competitive schools and they know more about the subjects they teach. Ms. Hannaway notes, "Students are better off being exposed to teachers with a high level of skill."

The strong performance in math and science seems to confirm that the more specialized the knowledge, the more important it is that teachers be well versed in it. (Imagine that.) No amount of time in front of a classroom will make you understand advanced algebra better.

Teach for America was pleased, but not exactly shocked, by these results. "We have always been a data-driven organization," says spokesman Amy Rabinowitz. "We have a selection model we've refined over the years." The organization figures out which teachers have been most successful in improving student performance and then seeks applicants with similar qualities. "It's mostly a record of high academic achievement and leadership in extracurricular activities."

Sounds like the way the private sector hires. Don't tell the teachers unions.

Friday, June 13, 2008

Mandated Tutoring Not Helping Md., Va. Scores

From: TheWashington Post
By
Maria GlodWashington Post Staff Writer Friday, June 13, 2008; Page B01
http://www.washingtonpost.com/wp-dyn/content/article/2008/06/12/AR2008061203681.html

Several States Find 'No Child' Provision Does Little to Improve Test Results

Free tutoring that federal law prescribes to help students at struggling schools has yielded little or no positive effect on student test scores in Virginia, Maryland and several other states, according to early evaluations.

Under the six-year-old No Child Left Behind law, certain schools in which too many students fail math or reading exams must use federal funds to offer after-school or weekend tutoring to students from low-income families. In the 2006-07 school year, $595 million went to the fast-growing industry of for-profit and nonprofit tutoring providers. But it remains unclear whether or how much those extra lessons are boosting student performance, even though the law envisions them as a key way to narrow achievement gaps.

In Virginia, researchers compared the performance last year of students with identical or very similar math scores in 2006 and found that those who were tutored did no better than their peers, according to an analysis the state Department of Education released in April. In a similar comparison of reading scores, students who were tutored lagged behind those who weren't.

Studies in Tennessee, Alabama, Georgia, Michigan and Kentucky also showed that the mandated tutoring, known as "supplemental educational services," didn't bump up test scores.

"This isn't helping poor kids," said Jack Jennings, president and chief executive of the Center on Education Policy in the District, which monitors implementation of the federal law. "All it's doing is taking money out of classrooms and putting it into the hands of private companies."

Jennings said that states don't have the capacity to monitor tutors effectively and that too many lessons aren't designed to build on the skills students learn in school.

In Maryland, students served by most of the state's tutoring providers in 2006 did not outperform students with similar academic profiles who weren't tutored. But students in three of the 29 state-approved programs did make bigger gains.

Steven M. Ross, executive director of the Center for Research in Educational Policy at the University of Memphis, which is conducting evaluations in Virginia, Maryland and several other states, said parents and educators generally give tutors good ratings. But, he said, "we're not seeing a big blip on the radar screen of raising standardized test" scores.

Ross cautioned that the assessments involve a relatively small sample of students. He said that tutoring might be helping them learn but that the help might not immediately translate into higher test scores. Some students who have fallen far behind, he said, could make progress but still fail grade-level tests. Or students might need more time with tutors.

"If I pour one gallon of gasoline in my car . . . I don't say it doesn't work if I don't go 100 miles," Ross said.

Turning to private tutors when public schools fall short is a key provision of the 2002 No Child Left Behind law. Under the law, schools that don't meet test performance goals for two consecutive years must allow students to transfer to higher-performing schools. Schools that fail to make progress for three years must offer private tutoring to children from low-income families. Those that continue to fall short face further sanctions.

As Congress considers revamping the law, the evaluations will fuel debate over whether tutoring is a wise investment.

Doug Mesecar, an assistant deputy secretary of education, said that officials remain confident of the value of the tutoring program but that more needs to be done to ensure quality.

"I think some providers are very effective as they work with students every day," he said. "The challenge we have is to figure out which ones they are."

Education Department officials point to a Rand Corp. study last year that found tutoring programs improved reading and math performance significantly in several large urban school systems.

In April, Education Secretary Margaret Spellings announced that she will use her administrative authority to promote participation in tutoring. Schools will have to improve outreach to parents about tutoring, and states will have more responsibility for ensuring that lessons meet students' needs.

Spellings also wants to require schools to prove that they've made an effort to recruit students into the tutoring before spending those funds elsewhere.

Nationwide, nearly 530,000 students -- 14 percent of those eligible -- participate, officials said. About 16,000 are in Maryland, Virginia and the District.

As schools work toward a goal of having every student proficient in reading and math by 2014, the number of children in tutoring is expected to rise. The number of providers has tripled, to more than 3,000, since 2003. Kaplan Inc., a subsidiary of The Washington Post Co., operates such tutoring programs in Maryland, Virginia and other states.

States have been slow to develop systems to gauge the effectiveness of companies and nonprofit organizations that work with students. Many schools report poor student attendance at tutoring sessions.

Some school officials say that even with those challenges, tutoring is making a difference. Chicago public schools found that students who were tutored outperformed peers in reading and math. Tutoring in Hawaii and Colorado has been linked to gains in math.

The District has not formally evaluated its tutoring programs, according to the office of State Superintendent of Education Deborah A. Gist. Evaluations are planned or underway in California, Texas, Florida and several other states.

n Maryland, where about 11,000 students were enrolled in tutoring programs in 2006-07, school officials say they support the effort even if it isn't producing big swings in test scores. The state spent more than $10 million in federal funding on tutoring last year.

"We see this is an opportunity for students to get ahead," said Maria Lamb, director of the Maryland State Department of Education's program improvement and family support branch. Lamb stressed that early evidence shows some students made gains.

Mrs. Dowd's Teaching Service was one of two Maryland tutoring providers linked to higher 2006 reading scores. (Another was linked to higher math scores.) Eileen Dowd, a former Cleveland schoolteacher, said her tutors work with, at most, three students at a time. Children who struggle, she said, get one-on-one attention.

Dowd's tutors work on campuses in Prince George's and Baltimore counties, enabling them to have close contact with teachers, she said. "It's collaborative," she said. "They will come and say: 'Tim is having a hard time focusing. What do you think?' "

Dianne M. Piché, executive director of the Citizens' Commission on Civil Rights, which supports No Child Left Behind, said the law gives low-income families access to a service that middle-class and wealthy families often use to give their children an edge.

Piché said that schools and tutors should work together more closely and that after-school help should be offered in places accessible to children.

"We need to push the schools and the providers to get it right," she said.

Staff researcher Meg Smith contributed to this report.

Wednesday, June 11, 2008

Teachers: Give us better tech training, support


New report reveals continued barriers to using technology for classroom instruction

After more than decade of investment in school technology, educators say they still don't feel adequately prepared to integrate instructional software into their classrooms and aren't getting the technical support they need to fully impact student achievement, according to a joint study by the nation's two largest teacher unions.

Released June 10 by the National Education Association (NEA) and the American Federation of Teachers (AFT), the study--called Access, Adequacy, and Equity in Education Technology--examines the state of educational technology resources and support in public schools across the country, as reported by classroom teachers and instructional assistants.

Although they often have access to computers and the internet in their classrooms, many teachers don't feel adequately prepared to use technology to enhance their lessons, the report suggests. What's more, many teachers in urban schools say they have insufficient or outdated equipment and software.

"Teachers and students should have the same level of technology in schools that is being used outside of schools. How can we expect our teachers to provide kids with the education they need to join today's high-tech workforce without the necessary equipment and training?" asked NEA President Reg Weaver.

The report shows that most educators use technology for administrative tasks, but substantially fewer use it for instruction. Although most educators believe that technology is essential to teaching and learning, they are less likely to use technology when the technology is outdated and has not been maintained. Educators also say they would like better support and technical assistance for using both software and hardware, especially in urban schools.

"When you see the overall condition of many of our schools and the support they receive, it is really not surprising that so many schools are lagging in technology," said AFT President Edward J. McElroy. "This is just one more indicator that policy makers need to set a much higher value on supporting our public schools and our students."

More than half of the educators surveyed said they had no more than two computers available for students' use in their classroom--and fewer than half mentioned their classroom as the main location where students work on computers for class assignments.

Elementary-level teachers have more computers inside their classroom for student use, but they are less likely to be satisfied with the software for their students and are less likely to have high-speed internet access in their classroom, according to the study.

Although three out of five educators said their districts require them to take part in technology training, respondents indicated their training has been more effective for non-instructional tasks, such as how to use the internet for research and how to use administrative software. Only 46 percent of educators believe they were adequately trained to integrate technology into their instruction.

Most teachers say their own access to technology at school is sufficient to do their job, but they reported getting little help with access to technology outside of school. Fewer than a third of those surveyed said their district has provided them with a laptop for planning and instructional purposes inside and outside of school, and only one-fifth said their district offered assistance for them to buy a computer for use at home (such as through low-interest loans, grants, or discounts).

The report urges policy makers to increase access to technology both in the classroom and outside of school by providing more wireless and portable technology.

It also recommends establishing standards for student usage to integrate technology deeper into the school curriculum; bolstering professional development by providing more appropriate training; increasing access to technical assistance; and engaging teachers' unions in planning for and implementing technology in schools.

The report is based on a survey of nearly 2,000 public school educators. It has a margin of error of plus or minus 3 percentage points, researchers said.

Link:
Access, Adequacy, and Equity in Education Technology

Tuesday, June 10, 2008

Obama & McCain Ed. Advisors at AEP

From: inService
http://ascd.typepad.com/blog/2008/06/obama-mccain-ed.html
Submitted by Naomi Thiers, Associate Editor, Educational Leadership magazine.

"Teachers are the intervention. Programs are not the intervention," Jeanne Century (of the University of Chicago's Center for Elementary Mathematics and Science Education) asserted when asked Obama's view on Reading First and good reading programs. Lisa Graham Keegan (of the Keegan Company) responded to the same question by stressing that rather than "waiting in the weeds" until one program is validated, teachers should follow key principles learned from Reading First—such as direct instruction and a base in phonics.

Century, an education advisor from Obama's campaign, and Keegan, an advisor for McCain, each spoke with passion and at times precision about their candidates' positions on education at the Great American Education Forum held June 6 as part of the Association of Educational Publishers conference in Washington, D.C. Both fielded questions from players in the media and education publishing world.

Sharp differences emerged when Joel Packer of NEA asked for each candidate's view on the balance between federal and state government in education policy. The important thing for McCain, Keegan said, is for states to take the lead in developing their own standards and accountability systems, possibly benchmarking against international standards. "The senator is not in favor of national standards," she asserted.

Century said standards are important to Obama, but "the federal-state balance is about more than standards." The federal government should play a key role in "making sure all parents have the resources to bring their kid to school ready to learn." Obama's plan, she stressed, includes measures to ensure all children have health care, high-quality childcare, and academic help beyond the school day.

To a question about merit pay, Century said Obama opposes tying teachers' pay to student achievement, but would consider rewarding teachers for a range of kinds of excellence, such as skill with specific populations or deep content knowledge. McCain, in contrast, believes teacher rewards should be tied to student performance, Keegan claimed.

Questions on NCLB and assessment also highlighted differences. Keegan said McCain believes we should explore "growth models" for measurement. Century stressed the need to reach beyond "rigid targets. . . . We should be assessing students' 21st century skills. Can students bring knowledge to novel situations? Can they use evidence?"

If schools are truly testing for 21st century skills, Century said, teachers can "teach to the test" in good conscience. What do you think about this statement? What parts of the candidates' education platforms do you agree or disagree with?

Monday, June 9, 2008

Friend or Foe? Balancing the Good and Bad of Social-Networking Sites

From: Digital Directions
http://www.edweek.org/dd/articles/2008/06/09/01networks.h02.html
Vol. 02, Issue Spring/Summer 2008, Pages 15,17,20
By Michelle R. Davis

Principal Conn McCartan of Minnesota’s Eden Prairie High School had no plans to police the Internet and its social spheres. But in January, he was mailed a computer disk containing photos of students drinking alcohol, and the photos had been posted on the social-networking Web site Facebook. McCartan couldn’t ignore the rule-breaking.

McCartan and his staff interviewed 43 students; 13 of them were subsequently disciplined. Most were members of athletic teams and clubs that have specific prohibitions and penalties for underage drinking.

“Facebook is a public site, but we didn’t go out there looking for it,” McCartan says of the misbehavior that came to light. “Somebody sends us something, and we’re obligated to respond.”

With the advent and rapid growth of social-networking sites like MySpace and Facebook, an increasingly significant portion of school-age socializing takes place online. The result is that school leaders are being forced to deal with a host of unsettled and even unsavory issues—such as when to monitor students’ online activities, and how to deal with the very real results of online socialization that spills into school hallways.

As educators begin to appreciate the influence these cyber networks are having in the world of teenagers, they are also mulling ways of drawing on that electronic muscle to forge a deeper educational connection between students and their studies. But experts say educators must have a clear vision and guidelines for doing so, or they will face serious technical and legal pitfalls. And, beyond those steps, experts say schools have a role to play in educating students about safely and appropriately using such sites.

“These things are very powerful,” says Tom Hutton, a senior staff attorney at the National School Boards Association. “More and more schools are realizing that it’s something we should find a way to tap into.”

'Very Little Precedent'

Students are using social-networking sites more than many school officials may realize. Despite the fact that most schools block access to such sites via school computers, 9- to 17-year-olds spend as much time using the Internet for social activities as they spend watching television—about nine hours a week, according to a 2007 study by the Alexandria, Va.-based NSBA. The study of more than 1,200 students found that 96 percent of those with online access had used social-networking technology—including text messaging—and 81 percent said they had visited a social-networking Web site at least once within the three months before the study was conducted.

More and more, those sites have become places where students engage in public actions or behaviors they probably don’t want their principals or teachers to know about. Students in New York City’s Staten Island borough were unmasked as graffiti artists earlier this year after posting pictures and video of their “tags” on MySpace and the video site YouTube. In York, Pa., 18 high school students faced disciplinary action after Facebook photos surfaced showing students with alcohol. The list of students nabbed for improper behavior through posts on social-networking sites reaches across the country.

Though most school administrators don’t spend their days trolling such sites for evidence of students’ unseemly actions—and don’t want to—they’re regularly faced with deciding when to follow up on tips or rumors.

“Does a school’s authority reach out to that wacky party on Saturday night that was documented by cellphones?” says Timothy J. Magner, the director of the U.S. Department of Education’s office of educational technology. “It’s clearly one of those evolving community dialogues.”

McCartan, the principal in the 10,000-student Eden Prairie school district, says even if school officials say they’re not going to monitor students’ behavior on the Internet, administrators need to be prepared when such a situation arises. At Eden Prairie High, school leaders had already talked with families about the importance of monitoring social-networking sites and discussed “what happens when we would receive this type of information and how we would react,” McCartan says.

“We thought this through ahead of time, and went through our own guidelines and consulted with legal counsel,” he says. “We knew if we proceeded this way with this information, that we were rock solid.”

Montana Miller, an assistant professor of popular culture at Bowling Green State University in Ohio and an expert on Facebook, says the issue remains complicated. Policies are still developing, and legal issues surrounding social-networking sites have not been settled.

“There’s very little precedent to go on, and it’s important to remember to take each case in its own context,” Miller says. “I do not think that schools should try to monitor and control Facebook the way they might try to monitor and control on-campus student expression in a traditional way.”

Miller acknowledges that the rules for school administrators are “murky,” but agrees that school leaders should discuss strategies for handling situations connected to social-networking sites before being faced with a dilemma.

“The key is that each school needs to have a discussion and come up with a policy, and then that policy has to be made clear to teachers, parents, and students,” she says. “Then that policy has to be followed. It’s really important not to play fast and loose with these policies, because that’s so hypocritical.”

From Virtual to Reality

All the cyber socializing by students in their after-school hours inevitably leaks into schools. In the 137,000-student Montgomery County, Md., public schools, Walt Whitman High School had two recent social situations that went from the virtual world to reality. In April, Whitman Principal Alan S. Goodwin handled two separate incidents in which taunting and name-calling on Facebook resulted in physical fights at school. Two of the students fighting were girls, and two were boys.

Goodwin put out an e-mail asking parents to closely monitor what their children were writing on Facebook and to consider calling the police if a student was being bullied online. A new Maryland law adds cyberbullying to the legal definition of bullying in the state and requires school boards to write anti-bullying policies by next year.

“One of the problems with Facebook is that people are more willing to say things there than they ever would to a person’s face,” Goodwin says. “If two kids are name-calling, their friends are on Facebook too, watching it. … They try to incite the situation.”

Many schools felt compelled to develop cyberbullying policies after the suicide last year of a 13-year-old Missouri girl, Megan Meier, who was the victim of virtual bullying through her MySpace page. Even so, some school officials still don’t understand the impact such harassment can have, says Miller, the Facebook expert.

“Never respond with ‘Just turn off the computer,’ ” she says. “That’s completely missing the essential nature of what it is to be a young person today. Facebook, MySpace, online communication, text messaging, and instant messaging are all integral parts of the social world of young people, and to tell them to turn off the computer is not the answer.”

McCartan says his approach to online bullying mirrors the one he’d take with verbal or physical bullying that occurs in person and off school grounds.

“We’ve been well served by reacting the same way we would to this information if it came to us any other way,” the Minnesota principal says. “Just because it exists online or in the electronic community instead of person to person, let’s apply the exact same standards.”

Hutton of the NSBA advises school leaders to first try mediating between the students and getting the parents involved. He says the same goes for situations in which school leaders may be alerted to pictures of students consuming alcohol or drugs, but in ways that don’t directly tie to school disciplinary infractions.

“School safety officers may be plugging in the name of their school [in an online search] and seeing what’s out there, but it doesn’t give them carte blanche to go after people,” he says. “We urge school boards and lawyers to talk through this and be very assertive about where your authority is.”

Social Networking TIPS

1. Establish a policy for dealing with incidents in which students break school rules and their inappropriate behavior is showcased publicly on social-networking sites.

2. Outline clear guidelines for administrators that spell out how schools should discipline students based on information garnered from social-networking sites, and let parents and students know about those rules.

3. Educate students about online-safety issues and how to use sites such as Facebook and MySpace responsibly.

4. Have a policy in place for dealing with cyber bullying.

5. If teachers are using social-networking sites for educational purposes, they should establish clear guidelines for how they intend to communicate with students via those sites.

Candidates Are at Odds Over K-12

From: Education Week
http://www.edweek.org/ew/articles/2008/06/11/41election.h27.html?tmp=1022169873
By
Alyson Klein and David J. Hoff
Vol. 27, Issue 41, Pages 1,19

But McCain and Obama Both Back NCLB Goals

The presumed November matchup produced by the long presidential-primary season that ended last week offers contrasting approaches to K-12 policy, along with some common ground on the basics of the No Child Left Behind Act.

Sen. John McCain of Arizona, the presumptive Republican nominee, and Sen. Barack Obama of Illinois, who last week secured enough delegates to claim the Democratic nomination, both express support for the NCLB law’s goals and its use of testing to measure schools’ success.

But Sen. McCain would promote market forces as a way to spur school improvement, and would likely seek to freeze education spending as part of a review of the effectiveness of federal programs.

Sen. Obama, meanwhile, promises to search for new ways of assessing students and to invest significantly in efforts to improve teacher quality.

Although education wasn’t a prominent issue in the Democratic or Republican primaries, it could emerge more clearly in the general-election campaign, one political scientist said last week. He pointed particularly to the potential for a sharper focus on where the candidates stand on the requirements for testing and accountability under the NCLB law.

In the past two presidential elections, the Democratic and Republican nominees supported the idea that the efforts to improve schools should include regular assessment of student progress and measures to hold schools accountable for increases in student achievement, said Patrick J. McGuinn, an assistant professor of political science at Drew University in Madison, N.J., who has written extensively about the politics of the NCLB law.

“The country hasn’t had a great debate about the costs and benefits of test-driven accountability,” Mr. McGuinn said. “We’re ripe for it right now.”

Changes in Testing

On May 28, in his most extensive education speech of the primaries, Sen. Obama reaffirmed his support for the goals of the 6-year-old federal law, saying they were “the right ones.”

“More accountability is right,” he said at Mapleton Expeditionary School for the Arts in Thornton, Colo. “Higher standards are right.”

But, Sen. Obama added, the federal government must provide enough money and other assistance to help substandard schools turn around, and he advocated improving the assessments that are the cornerstone of the law’s accountability system.

“We also need to realize that we can meet high standards without forcing teachers and students to spend most of the year preparing for a single, high-stakes test,” he said.

During the primaries, Sen. Obama never criticized the NCLB law with the same ferocity as Sen. Hillary Rodham Clinton of New York, his leading opponent for the nomination, or other Democrats candidates. Mr. Obama was not yet in the U.S. Senate when Congress passed the bipartisan measure in 2001 at the urging of President Bush.

‘Good Beginning’

Sen. McCain has said that he considers the NCLB law a “good beginning.”

He adds that the law needs to change to improve the way special education students and English-language learners are assessed. But he hasn’t suggested that he would change the way schools are held accountable for student performance under the law, which requires reading and mathematics tests in grades 3-8 and once during high school.

Sen. McCain has also said he would endorse federal programs that give parents broader school choice, such as vouchers for private schools, including religious schools.

Such initiatives are popular with Republican voters, but Sen. McCain’s strong support for the NCLB law isn’t widely endorsed by voters from either party, said one former Bush administration official.

Sen. McCain “is putting himself in a difficult situation by embracing NCLB so wholeheartedly,” said Michael J. Petrilli, the vice president of national policy and programs for the Thomas B. Fordham Foundation, who served in the U.S. Department of Education during President Bush’s first term.

“Everybody has acknowledged that the law needs some reworking,” Mr. Petrilli said, “and he has created this huge opening for Senator Obama, who can now embrace the ‘mend it, don’t end it’ platform, which is going to sound like the common-sense platform.”

Union Action

Still, some important new supporters of Sen. Obama will likely be urging him to recommend significant changes to the law.

On June 4, the day after Sen. Obama said he had the delegates to secure the nomination, the National Education Association announced it would endorse him in the general election. The 3.2 million-member teachers’ union is one of the most vocal critics of the NCLB law’s emphasis on testing.

While the NEA waited until Sen. Obama had essentially locked up the nomination before making any endorsement, the 1.3 million-member American Federation of Teachers was an early supporter of Sen. Clinton and worked actively on her behalf.

Because neither national teachers’ union supported Sen. Obama during the primaries, he may have the opportunity to be a “different kind of Democrat,” said Joe Williams, the executive director of Democrats for Education Reform, a New York City-based political action committee that contributes money to Democratic candidates.

“He’s earned his independence so that he can really decide which of the unions’ positions he really wants to embrace and which ones he doesn’t,” Mr. Williams said. “The conventional wisdom is the time that you’ve got to pander to the unions is during the primary. He emerged victorious without [their help].”

For More Info
For more stories on this topic see No Child Left Behind news page.
For more information and analysis on education and the 2008 presidential campaign, read Campaign K-12.