Abstract: Various ways of polling statistically significant numbers of members are examined and an attempt is made to assess them against the criteria of set-up cost, operational cost, access to and of the membership, likely response levels and speed of response. The author recommends a polling system based on regular succession of poll questions on membership renewal forms, collated by the registry over a polling period of 12 months.
Disclaimer: The purpose of this paper is to stimulate discussion on the ways the Board may poll the membership rather than to present a conclusion. It has not been possible to gather objective data on the efficiency and cost of the various proposals examined here, and at this early point in the process such is probably not necessary.
Assumptions: All proposals made in this paper are based on two key assumptions.
Any polling system recommended by the Grand Council should be assessed against the following criteria:
Set-up Cost: All other things being equal, the Board would prefer to use systems already in place rather than create new procedures and offices. The more expensive set-up is, the less likely the Board is to adopt the system. The ideal system would cost nothing to put in place.
Operational Cost: In order for any polling system to contribute usefully to the SCA, it must be used regularly. The more it costs for a question to be polled, the less likely the Board is to seek the feedback a poll would give. The ideal system would cost nothing to distribute questions, collect and collate responses and present results.
Membership Access: People cannot answer questions they never hear. Responses collected from forums where only some members have access have a risk of bias that may make any results statistically unreliable. The ideal system would reach every member.
Likely Response Levels: The more people who respond to a poll, the more accurate the results. The ideal system would extract a response from every member. In addition, the larger the number of responses, the more moral force the poll is likely to have.
Speed of Response: Surprisingly, this is not a major issue. Polls are requested by the Board, which meets every three months. Even if meaningful results were possible within a week of the polls opening, nothing could happen about the issue until the next Board meeting at the earliest. In addition, polls in the SCA have nearly always dealt with matters of group philosophy and issues that have been kicked around for years. Polls will not be run on operational matters that require rapid decision-making. Polling periods of months are acceptable if the delay is likely to produce a more reliable result.
Option 1 -- Request for comment in TI (current system).
The Board announces in TI that they are after member input on a particular question and request submissions that answer the question in a meaningful yes/no way.
Set-up cost: zero, because it is already in use. Even if it was not, the cost of one page every year or so in TI is obviously acceptable to the Board.
Operational cost: effectively zero, assuming that whoever collates the responses is a volunteer.
Membership access: almost total, because nearly all members get TI.
Response levels: pathetically low. In order to produce a result a member must read the Board notes in TI, formulate an opinion, write it down, address an envelope and post it. Not only does this require a degree of political commitment (about 10 minutes worth) from a population that seems to be rabidly opposed to political activity above the kingdom level, but I suspect it tends to attract more critical responses to a question than supportive ones. A person irritated by a proposal is more likely to dash off a letter than one who is approving. I believe average response numbers are fewer than 100, out of a population in the tens of thousands. With one exception (Jafar) all have tended to the negative on each issue.
Speed of response: adequate. I believe poll turnaround under this system is about six months. Nobody has ever criticised the polling process for being too slow, which indicates to me that speed is not an essential feature of any system we adopt.
Overall assessment: cheap, easy and worthless. Response numbers are not statistically significant, and I suspect are skewed to the negative. On the positive side, this polling system would suit a board that would like to be perceived as responsive to change without actually risking a compelling directive to change.
Option 2 -- Professional phone polling.
We contract with a professional polling organisation to run polls on matters of interest.
Set-up cost: zero.
Operational cost: unknown, but probably non-trivial, and would be incurred every time we run a poll. This is likely to discourage the running of frequent polls, which reduces the amount of member feedback to the Board, and is thus a Bad Thing. Before we can seriously discuss this matter, we need to get some quotes from a US based polling company.
Membership access: total in theory, as their sample would be drawn from the registry database. I have no idea what percentage of the membership would need to be included in a statistically valid sample, but I assume it is on the order of 500. At 5 minutes each and $10.00 per man-hour that comes to a bit over $400.00. Those figures are out of the air, as I have no idea of what these people charge, because (a) I am not a statistician and (b) Im in Australia.
Response levels: probably a meaningless issue in this context.
Speed of response: easily adequate, probably on the order of a couple of weeks. A report requested at one Board meeting could easily be presented at the next.
Overall assessment: depends entirely on how much this will cost. It seems perfectly adequate in other ways for our purposes, and is a reliable tool used in business and politics. However, I state a personal preference for techniques that produce massive numbers of returns. I want the results of polls to have moral authority, so they are not easily put aside, and I am not sure responses in the low hundreds will have this moral force. I would prefer responses in the thousands.
Option 3 -- Internet polling
Issues are placed on the Rialto and the various kingdom e-mail lists with a request for a response. Replies are sorted and counted automatically by some simple mechanism.
Set-up cost: probably trivial in the simplest form, but the more bells and whistles that are added to the system to more programming will be required.
Operational cost: trivial to zero. The only cost I can see is the ISP hosting charges.
Membership access: high, but not universal, and there is a real possibility that the population is not representative. In general terms the members reached by this system are students, professionals who use a computer at their work and people who routinely log on from home for recreational purposes. We have a large number of members outside those groups. I cannot predict if the skew is significant or would affect the results but the issue should be carefully assessed.
Response levels: this system could easily return hundreds or thousands of responses -- more than enough to be both statistically valid and morally weighty. However, quality of response may be an issue. E-mail is not a medium that encourages careful thought before clicking the mouse. It may be wise to institute a system where an attempt to debate an issue on all mailing lists is made for a couple of weeks before the voting site is activated, in the hopes of achieving something resembling an informed response.
Speed of response: assuming the simplest system, and depending on the degree of member interest, significant responses could be gathered within minutes to hours of the question being posted. However, given that the Board cannot consider the results until the next meeting, this speed is not useful.
Overall assessment: the cheapest and easiest way to get large numbers of responses to a poll. The major problems are likely to be the lack of easy access to the net for a large proportion of the membership, and the vote without thinking possibility of a one-click polling system. In addition, this system is the most vulnerable to manipulation. It may be wise to consider systems that will flag large numbers of multiple votes from the same IP address, or possibly uses membership numbers as PIN numbers, or some other security system. This will in turn increase the set-up cost of this system.
Option 4 -- Polling on membership form.
Poll questions are placed in membership application / renewal forms for months at a time and the results tallied as memberships are processed.
Set-up cost: Probably non-trivial, but not too bad. At the least, the registry database would need to be modified to hold a few extra fields, with associated modifications to forms.
Operational cost: A poll question on the membership form would add two extra keystrokes to data entry, assuming the entry dorm was designed efficiently. While this would mean that processing registrations would take slightly longer, the cost is probably trivial.
Membership access: 100% by definition.
Response levels: If a poll question is run over 12 months membership access is close to 100%, excluding half of those who pay biannually. Even assuming very large figures for the number of old and obsolete forms returned by members, and even assuming polling periods as short as 2 months, this system seems to offer very large numbers of responses evenly distributed across the membership demographic. It seems to me that it would be both statistically valid and have the moral force of numbers.
Speed: Assuming a three month polling period, and assuming membership processing takes 30 days, poll results would easily be available within six months.
Overall assessment: This is in theory my personal preference. The only practical issue is that the poll results are all channeled through the Milpitas office. Being an Australian I have never dealt with this office, but it has a reputation of being resistant to change and hard to work with. This system cannot work without the cooperation of the registry.
Option 5 -- Polling at events.
The Board asks the kingdoms to make petitions for and against issues available at events.
Set-up cost: as far as I can see, zero.
Operational cost: Effectively zero, both because any likely costs are very small, and because the kingdoms and not the corporation absorb them.
Membership access: Assuming a member goes to at least one event every three months, and that the petitions are at every event, then over a three month polling period every member should have at least one opportunity to poll.
Response levels: Totally dependent on the amount of advertising done by the kingdom. If the heralds at every event are required to call attention to the petitions and encourage the populace to respond, response levels would probably be high enough to be useful. If the forms are just on the troll table and never pushed by the crown, probably much less.
Speed: If a poll is run over three months, the results should be available within six. This system seems likely to be able to meet that.
Overall assessment: Attractive, because of the high member penetration, and because the collation of results could be delegated to the kingdoms as well, thus reducing the workload at the corporate level. However, this approach requires the kingdoms to both cooperate with the Board and treat the issues in an evenhanded manner. A king who uses his courts to critique a particular side of any poll issue and then encourages his populace to vote could skew the results a bit. In addition, there is the possibility of either accidental or deliberate multiple responses by individuals, and it may be necessary to check for such. This would significantly complicate the job of poll counting.
However, this system is almost identical to the current polling system used to determine changes in group status, and thus seems to be acceptable to the Board already.
Option 6 -- Postal polling.
All members receive a letter with a ballot to mark and return. The letter can be accompanied by briefing sheets setting out the case for and against the proposition.
Setup cost: trivial to zero.
Operational costs: significant. A mailing to each member at an estimated 50 cents total cost would be on the order of at least $10,000.00. If a postage paid envelope in included, that could double. (Question - what is our total membership, currently, and what is standard US postage?) In addition, for a poll of the total membership we would probably require counting at the kingdom or smaller group level rather than at one address. This significantly increases the difficulty of the process.
Membership access: 100%
Response levels: unpredictable, depending on the emotiveness of the issue. This is the system used in polling for group status change from principality to kingdom. It took Lochac three attempts at polling before we got more than 50% response, although we all had to lick our own stamp. Use of paid return envelopes may increase that response. The recent experience in Lochac is that organizing such a poll is a significant effort, and we are a small group. Doing it across the whole society at once is interesting to consider.
Speed: How fast do you want to go? My experience is that kingdom polls are preceded by a few months of discussion on the issue, followed by a mailout that must be returned within a month. That suggests a time frame of between four and five months before the results are known. However, I don't see high speed as a necessary requirement of any system we recommend.
Overall assessment: This system evidently works on any group up to the size of a small kingdom, because all groups have to go through it to become a small kingdom. As far as I know, it has only ever been used for group status polls, which are emotive issues with usually a lot of positive support. They are expensive to run, which makes them unattractive for frequent use. They are familiar to most of the membership, which is a point in their favour. On a society wide level, it would probably be necessary to devolve vote counting to the kingdoms or smaller regional groups, which makes the whole process organisationally complex.
Conclusions.
No matter how I look at the issues, a poll question on the membership form over 12 months seems to be the best system. Response numbers would be large, and samples would be unbiased by access to technology. Set-up and operational costs seem likely to be low. The major problem is one of management. Would the Milpitas office be willing and able to cooperate with such a system?
As I have thought about this, it has also occurred to me that for a poll to be useful, the populace must be informed of the issues and the arguments. This poses problems given a populace that on the whole doesnt give a damn. I would suggest for consideration that any issue put to poll by the Board be preceded by several months discussion on e-lists and in newsletters. A short-term Board subcommittee tasked with raising member consciousness of the issue and its various sides should stimulate this discussion. In other words, we dont just run polls - we market them. The logical place to find the committee members are the partisans who brought the issue to the Board in the first place, and those people who scream the loudest when the issue is first raised.
Please send comments to the Directors at the corporate
PO Box or to
directors@sca.org,
the Grand Council at
SCAGC-L@LISTSERV.AOL.COM,
or, better yet, both. You may send postal mail for the GC to:
Cyndi Baskett, P.O. Box 3460, University, MS 38677 (USA)
Janna G Spanne, Nyckelkroken 50, 226 47 Lund, Sweden,
Fax: +46 46 2224531 attn.: Janna G Spanne
You are both welcome and encouraged to post or reproduce this proposal, provided you do so in full without alteration.
For more information on the SCA, see the SCA Home Page
Last changed: 3-Apr-2006
Copyright of this site is retained by the original contributors. Contact webmaster prior to using elsewhere.