Imagine trying to show a bone fracture to someone using only words – no more than 250, with no Xrays, graphs, or other visual images. That’s what members of the American Academy of Orthopaedic Surgeons (AAOS) struggled with when submitting abstracts for AAOS’s Annual Meeting – until the organization’s program committee rallied for change.
“[They] demanded it. They said they weren’t getting what they needed and couldn’t go further with education until we changed our system,” said Kathie Niesen, CMP, AAOS’s education manager. “With no visuals and being limited to 250 words, it was too hard to judge the abstracts and it limited our ability to judge what the final presentations were going to be like.”
A measure of acceptance
Dramatic change came in the form of a partnership with Chicago based Coe-Truman Technologies, which allowed AAOS to overhaul its submission process and begin accepting abstracts with “images, graphs, X-rays, pretty much anything,” Niesen said. “And we are no longer limited to 250 words. The limit we decided on is one page, which is more than 2,000 words.”
The result? A record number of abstracts were submitted for AAOS’s 2012 Annual Meeting in San Francisco next month— 5,369 in all, nearly 300 more than were submitted for the 2011 Annual Meeting in San Diego. The abstracts also were of “higher quality,” Niesen said, “which is what the program committee was going for, because the graders could see what they were only able to read before.”
An added benefit of AAOS’s new submission process was “squeaky-clean” financial disclosure—in terms of participants reporting their potential affiliation with any pharmaceutical, medical- device, or publishing company. AAOS requires all Annual Meeting participants submitting abstracts to disclose certain financial information “for the purpose of the graders,” Niesen said. “The graders need to see each participant’s disclosure in order to grade the abstract correctly. The authors’ names don’t appear, but their disclosure does.” (See “The AAOSOrthopaedic Disclosure Program,” at bottom of page.)
For many years, AAOS sent financial-disclosure forms to authors and co-authors via snail mail. “The follow-up was onerous,” Niesen said, “and the storage requirements a nightmare, because we had to keep the forms for six years.” About four years ago, AAOS decided to get “ahead of the curve” and created its own financial-disclosure database. Members submitting abstracts were asked to log on to the database once a year and update their financial disclosure. Authors also were able to disclose on behalf of their co-authors. Once their abstracts were accepted, however, all authors and co-authors were required to disclose individually.
“In [the fall, with the Annual Meeting approaching,] we would be faced with 1,600 disclosures still out and we’d be chasing them literally all over the world, … because the United States is the only country where disclosure is required,” Niesen said. “Once, we tracked down a guy on an island. He was taking his dream trip around the world, and we had to locate him by satellite phone.”
AAOS’s partnership with Coe-Truman has streamlined disclosure by making it part of the actual abstract-submission process. For the 2012 Annual Meeting, authors submitting an abstract had to make sure both their own and their co-authors’ financial disclosure was complete before the abstract could be accepted.
New and Improved
To submit an abstract under AAOS’s new system, members log on to the organization’s website and are directed to Coe-Truman’s website to fill out the abstract. They then return to AAOS’s page to find their disclosure information—moving seamlessly between the two sites. “You find yourself in the system and make sure your disclosure is up-to-date,”Niesen said. “Then you find your co-authors’ disclosures. If you find John Brown’s disclosure and it’s not current, the system will send him an email reminding him to update his disclosure. Then you’re done with disclosure.
“You go back to Coe-Truman and submit the abstract. You get an email saying you are number 1,000 and we are waiting for John Brown to complete his disclosure. Every week you get an email saying John Brown still hasn’t disclosed; you might want to get in touch with him. Or it could say John Brown has updated his disclosure and the abstract is completed and ready for grading.”
The deadline for abstract submission for the 2012 Annual Meeting was June 1, 2011. “We gave a grace period until July 1 for the purpose of disclosure, so if the abstract is in the database, you have 30 days to get your co-authors’ disclosure,” Niesen said. “A lot of members were [medical] residents when they did their research. So it can be difficult to track down your co-authors. People have often moved to different hospitals or different countries.”
Although AAOS was “very pleased” with its new system and “optimistic for the future,” Niesen said that the transition was “rough.” For starters, “you needed to know your AAOS user name … or enter the right information to get your user name. To complicate things, we changed databases a year ago. So everyone had to have a new user name. A few thousand people didn’t have one; they had to be assigned one. There were a lot of emails and a lot of phone calls.”
Members also “duplicated themselves” in the database, creating another challenge. “We had to go into the database and try to figure that out,”Niesen said. “A lot of manual labor went into it—calling people trying to figure out if John Brown in Arlington is the same as John Brown in Dallas. Is one his work address and one his home address?”
Another hurdle was the “lack of understanding of foreign guests to the disclosure process,” according to Niesen. “It was really confusing and irritating to them. They don’t know why you need this information. They may think they disclosed, but they really didn’t. Or they may call you and say they have nothing to disclose and think they don’t need to do anything else.” One item on AAOS’s to-do list for its 2013 Annual Meeting is optimizing search results in its database. For 2012 “we tried to match on name and email address,” Niesen said. “But that didn’t work because email addresses change all the time. Now we’re going to do name, email, city, state, country—any or all of those. It’s a wider parameter of search, so someone searching for co-authors will have more options.”
‘Lessons Were Learned’
In the end, the new submission process “took much more staff time and much more staff this time around. The membership department, switchboard, customer service, and almost all of the people in the convention and meeting-services department were helping out at one time or another,” Niesen said, noting that AAOS has already begun talking to Coe-Truman about next year’s abstracts. “The meetings we are having with Coe-Truman and our IT department will make the process seamless in the next few years.”
Susan McSorley, AAOS’s director of convention and meeting services, agrees. “Coe-Truman has been a good partner,” she said. “Lessons were learned in communication with submitters since this was a new process of submitting an abstract as well as disclosing for the presenter and all co-authors. We have identified and worked through the areas that created challenges and look forward to utilizing the system for next year.”
Coe-Truman also was “very happy with the way [the project] turned out, despite any hiccups along the way,” said Jon Jenkins, Coe-Truman’s director of product management and development, who was one of the senior architects of the AAOS project. “Any time you go through the same process more than once, you drive efficiency. I would be very worried if we didn’t. If we got it perfect the first time I would argue that we are not working hard enough to improve.”
Picking Your Partner
Kathie Niesen, CMP, education manager for the American Academy of Orthopaedic Surgeons (AAOS), has one piece of advice for medical associations interested in upgrading the abstract-submissions/disclosure process. “Make sure you have the right vendor,” she said. “I can’t tell you how important that is. Make sure your IT department and the vendor have a relationship and work together. I could never have done this without both of them.”
AAOS put its abstract-submission process out to bid and got a “wide range” of responses. “We felt Coe-Truman had the staff and professionalism and willingness to make it work,” Niesen said. “They did whatever they had to do to make it work. I think they bit off more than they could chew, and we did, too. But we made it work by working together. It really was a collaborative partnership.”
Coe-Truman went into the partnership with “respect” for AAOS’s internal system, according to Jon Jenkins, Coe- Truman’s director of product management and development. “They have a system that is very central to what they do,” Jenkins said, “and they have a very capable tech team on their end. We have to honor that as a vendor and not impose our system on them. If they have a good system that works, we need to integrate with that.” He added: “You can’t say there are only one or two ways to do something. If you’re flexible, and almost plan for that flexibility, the final outcome will be much more effective.”
The AAOS Orthopaedic Disclosure Program
Each item below requires an answer. You will be prompted for more information after you have answered all of the questions with a “yes” or “no” response. Please report information for the last 12 months as required by the Accreditation Council for Continuing Medical Education (ACCME) guidelines. After submitting your Yes/No answers, you must visit each detail page before you may confirm and save your disclosure. If your answers are accurate, you may return to the summary page and then confirm and save your disclosure.
* indicates a required entry
*1 Do you or a member of your immediate family receive royalties for any pharmaceutical, biomaterial, or orthopaedic product or device?
Yes I No
*2 Within the past 12 months, have you or a member of your immediate family served on the speakers bureau or have you been paid an honorarium to present by any pharmaceutical, biomaterial, or orthopaedic product or device company?
Yes I No
*3A Are you or a member of your immediate family a PAID EMPLOYEE of any pharmaceutical, biomaterial, or orthopaedic device or equipment company, or supplier?
Yes I No
*3B Are you or a member of your immediate family a PAID CONSULTANT for any pharmaceutical, biomaterial, or orthopaedic device or equipment company, or supplier?
Yes I No
*3C Are you or a member of your immediate family an UNPAID CONSULTANT for any pharmaceutical, biomaterial, or orthopaedic device or equipment company, or supplier?
Yes I No
*4 Do you or a member of your immediate family own stock or stock options in any pharmaceutical, biomaterial, or orthopaedic device or equipment company, or supplier (excluding mutual funds)?
Yes I No
* 5 Do you or a member of your immediate family receive research or institutional support as a principal investigator from any pharmaceutical, biomaterial, or orthopaedic device or equipment company, or supplier?
Yes I No
*6 Do you or a member of your immediate family receive any other financial or material support from any pharmaceutical, biomaterial, or orthopaedic device or equipment company, or supplier?
Yes I No
*7 Do you or a member of your immediate family receive any royalties, financial or material support from any medical and/or orthopaedic publishers?
Yes I No
*8 Do you or a member of your immediate family serve on the editorial or governing board of any medical and/or orthopaedic publication?
Yes I No
*9 Do you or a member of your immediate family serve on the board of directors or a committee of any medical and/or orthopaedic professional society?
Yes No
Reprinted with permission from AAOS.
MORE RESOURCES:
For more information about AAOS’s 2012 Annual Meeting, visit http://bit.ly/AAOS
When the American Academy of Orthopaedic Surgeons overhauled its online abstract-submission process, the result was better abstracts – and ‘squeaky-clean’ financial reporting.
Imagine trying to show a bone fracture to someone using only words – no more than 250, with no Xrays, graphs, or other visual images. That’s what members of the American Academy of Orthopaedic Surgeons (AAOS) struggled with when submitting abstracts for AAOS’s Annual Meeting – until the organization’s program committee rallied for change.
“[They] demanded it. They said they weren’t getting what they needed and couldn’t go further with education until we changed our system,” said Kathie Niesen, CMP, AAOS’s education manager. “With no visuals and being limited to 250 words, it was too hard to judge the abstracts and it limited our ability to judge what the final presentations were going to be like.”