During the last couple of years there has been a lot of conversation, criticism and debate about the nature of pattern testing within the blogosphere. And I understand why.
After reading this great piece by Sue (who happens to have recently tested a pattern for me) over at Sewin' Steady (as well as the comments that follow), I decided I wanted to get involved in the conversation in the hope that I can also encourage it to be discussed (and even questioned) in a positive and constructive way.
Pattern testing really is a difficult thing to talk about (I have drafted this blog post at least three times and I’m still not sure it conveys exactly how I feel about it.) as it is a murky area.
What is pattern testing?
For those of you who are unfamiliar with pattern testing, this is how it works. The transaction normally goes something like this. Designer creates pattern. Designer puts a call out on blog and or social media asking for people to apply to test said pattern. Designer selects a group of testers. Designer sends pattern to pattern tester. Tester makes garment using pattern. Tester is paid in “kind" (usually a free pattern) for volunteering their time. Tester (generally) then reviews garment on their blog. Designer uses tester images to promote pattern.
The main issues I have seen discussed surrounding pattern testing and the things I would like to talk about are:
1. How are pattern testers selected?
2. Is pattern testing a means of actually testing a pattern, or is it a promotional tool?
3. How do we encourage "real" feedback?
5. Is it unethical that testers are not compensated financially for their time / work?
1. How are pattern testers selected?
I have seen this question come up a lot as there are many people out there who are applying to test over and over and never getting selected. I find this question quite difficult to answer as I don’t really have a proper strategy for selecting testers. Well at least, not yet.
When it was time to test my first pattern, the Rushcutter, I didn’t have a website for In the Folds (only my old site Em Makes Patterns) or and Instagram account, so I decided to put a call out for testers in my previous site as well as a few Facebook groups. I was blown away by the response. I received around 120 applications in the first day. I was most surprised by this response because I was essentially a nobody. I had not released a pattern before, and had no online presence, yet so many people were willing to test a pattern for me when there was no guarantee that the pieces would even fit together. I was so touched by the generosity of all these women that I wrote an email to every single one of them thanking them for applying.
I was then stumped by the question of how to choose who I would ask to test the pattern. I started by putting all the applications into a spreadsheet and then going through and making sure I had at least two testers to select from for for each size. The dress can be made in two different styles and I had asked for preferences in the application, so I used this as a way to select testers. If only one person who wore a size C wanted to make view B, then that is who I selected. I then removed the testers who identified as beginners. I did this because the Ruschutter is not designed for beginners and I did not want anyone to feel out of their depth. Testing is supposed to be fun, and I didn't want to leave people feeling upset or frustrated because it was too challenging. I also didn’t want to select all the advanced makers who applied as I really wanted to know how a diverse group of makers would feel about the pattern.
I knew that I would not be doing a shoot for the Rushcutter at the time, so I also aimed to choose people who I knew would take a clear photo of the front and back so that I could include it in the product listing (I worked this out by looking at people's blogs or the photos they sent through Flickr, dropbox or on Facebok pages). I wasn't looking for over the top shots, just photos that clearly highlighted the garment details.
I felt much calmer when it was time to test my second pattern. For one, I’d done it all before and had a good idea of what to expect. I also had a number of women who I had established a relationship with the first time around that I knew would jump on board again. This is something I have seen / read criticism about on a few blogs. I know many people have mentioned a bit of a “clique” when it comes to testing - feeling as though often the same people are chosen over and over to test patterns. I do understand that, but speaking from experience, I also know I felt much more comfortable asking people who had by this point become friends to help me out by volunteering their time, rather than asking a whole lot of new people to donate their time to my project. By establishing and then building an ongoing relationship I feel it is much easier to return the favour eventually, which I hope I can find a way to do with as many of my testers as possible. In saying this though, I did ask if people had tested before in the application so I could ensure I introduced some new testers into the pool and could also give them a chance. Roughly one third of the women who tested my latest pattern had not tested before and about half who tested the Ruschutter.
This time my selection method was slightly different. I decided to choose people, when possible, that had purchased the Rushcutter. I want testers who are going to enjoy making my patterns, who are actually going to wear the dress they make with the pattern. I don’t want the dress to end up in a wardrobe and never looked at again. By selecting women who had purchased the Ruchcutter, I felt that they already identified with my design aesthetic, my brand resonated with them and they were likely to enjoy making and wearing the Acton too. I also saw it as a way to thank them for supporting me and my little but growing brand.
I am still not sure of the best way to select testers and will be working on this each time I test a pattern, but for the moment I am learning as I go.
2. Is pattern testing a means of actually testing a pattern, or is it a promotional tool?
This is a difficult question and I am obviously not in a position to answer this for everyone. But I am in a position to answer for myself and I think it is important that I do so.
The reason I test patterns, with help of volunteers from the sewing community, is most definitely to see that the pattern works in its entirety. Marketing does not come into the equation. I don’t insist on testers having blogs to apply to test, nor are they expected to write a review of the pattern after its released. The decision whether or not to review the pattern is totally up to them.
The reason I test a pattern is because I want to be sure that my pattern is an excellent product. When I send out a pattern for testing I ensure it is a darn good product before it even arrives in a testers inbox.
When I send it out I am sure of many things - I am sure the pieces fit together (because I check each piece from each size), I am sure it prints well (because I print the pattern at home and make sure that it does), I am sure that the techniques used are the best choices (because I make around ten toiles of a given design before I even start the grading process), I am sure it is graded correctly and I am sure that the instructions are clear and written well (because I spend a huge amount of time making sure that they are). It is really important for me to be sure of these things because I do not want to send out a pattern that is a flop, whether it is in a ‘testing period’ or not. I do not want people to waste their precious time, paper, ink and fabric to create a garment that just cannot be worn because of an error on my end.
Although I am sure of a lot when I send a pattern out, I still want to see a group of very different people approach the same pattern and see if it sheds light on anything that can be improved. Testing provides me with an opportunity to see a dress made in every size I offer (although I could go and make a dress in every single size, I would struggle to find ft models of all the sizes in my range) by women who have varied levels of skills. I am an advanced level sewist and can often underestimate how difficult a process is. Having testers of all different skills helps me with this immensely. No matter how much I scrutinise a pattern, there are simply things that I do not see. Beginners notice things that the more advanced sewers don’t see and vice-versa.
As well using testers of different size, shape and sewing skill, I also use testers from all over the world. This helps greatly and ensures that people can access the things they need to make the pattern in their area. For example, when I tested the Rushcutter I simply converted the button measurements from centimetres to inches, as I assumed this was how it was done. One of my testers highlighted that buttons of the size I specified could only be purchased in specialty shops in the US. I knew it was important that I changed it so it would be no trouble for those sewing the pattern. It is also important to check that all the testers also understand the terminology used in the pattern. For example, I used the word “drill hole” a number of times in my most recent pattern and one of my testers (thank you Erin) pointed out that this was not a term she knew. I then decided to put it in the glossary of terms. These are the type of things I would not be able to work out myself and what I am very thankful for my testers for. These things ensure that users of my product have a great experience using the pattern when it gets released. There is no frustrating moment when they realise they cannot find the button they need or have to scour the internet to fond the meaning of a term I use.
So in short, for me the point of testing a pattern by asking people from the community is to confirm that I have created a good product, to iron out any creases and to see how a wide range of women approach the pattern. I do not rely on my testers for reviews. After the Rushcutter was released, I contacted people directly and asked if they would like a free pattern in exchange for a review. In this case it is important that the reviewer highlight that they have not paid for the pattern.
3. How do you encourage "real" feedback?
I ask testers to be as open and honest with their feedback as possible, specifically asking what they didn’t like about the pattern in the feedback form. I believe this is the best way to extract constructive feedback from people (because generally people are nice and don’t want to say ‘nasty’ things). I would definitely prefer to hear what’s wrong with the pattern before it’s released, when I am still in position to fix it, rather than afterwards what it would be very difficult (as well as awkward and embarrassing) to call it back. I occasionally see designers releasing “new and revised” editions of patterns. I really don’t want to go down this road. Updating and refreshing instructions or format is one thing, but changing the actual pattern pieces? All the creases should really be ironed out before the first pattern is sold.
4. When a pattern is reviewed after testing, is it expected that it has to be positive?
I think its fantastic when testers choose to review a pattern after it’s release. When testing for me, testers are under no obligation to write about the pattern, nor are they under any obligation to give it a positive review. I think honest reviews are what keep things interesting and are often what sell patterns.
I do appreciate that it is often very difficult to say anything negative about a pattern that you know the designer has spent a huge amount of time working on it. Indie pattern companies are not faceless. In most cases we know exactly who is behind the design, how hard they worked on it (and even sometimes what they had for dinner! Thank you Instagram.) This is a harder one to tackle as no matter how many times you encourage people to be open and honest with their review, it is a difficult thing to do. In a sea of generally positive reviews, it is always difficult to be the one to point out the flaws. It is also difficult to feel as though you may upset the designer.
I did not do a blog tour when I released the Rushcutter, but have been toying with the idea for my next release. But thinking about the nature of reviews and how it sits with testing, has made me think that it may not be the right things to do. Would by encouraging people to take part in a blog tour make them feel as though they had no choice but to give the pattern a good review? Or is it a good way to say thank you to the testers, as surely a few people in the testing pool are hoping to get more traffic to their site by testing. And here I am going around in circles. What do you think about this?
And onto the last question in this piece.
5. Is it unethical that testers are not compensated financially for their time / work?
I have read a few posts about this that pretty much accuse designers of exploiting people to get their patterns tested. I hate reading stuff like this as it really is so far from the truth (well at least it is for me and the way I do things) and it makes me feel very sad. It gives no credit to the incredible relationships that are formed during the testing process, the fun that people have and all that is learned (by both designer and tester) during the testing period. I guess what I am saying is that it gives no credit to the sisterhood that exists in our incredible community.
Choosing to test a pattern is a choice. Women (and maybe in some cases men) nominate themselves and volunteer their time and their skills to get involved in the process for a million and one reasons. They may want to see how it works, "meet" other makers, contribute to the community, learn something new, have a deadline, restore their "sew-jo" and the list goes on… I have tested a pattern before. Do you know why? Because I knew I wanted to start an independent pattern company at some stage and I knew I could learn from the testing process. And I did. I learned lots! And receiving a free pattern was just a bonus at the end.
This does not mean that I think it's good that testers are generally not paid. I, as much as any other designer out there, would love to be able to pay people to test my patterns. But at the moment I am not in a position to do so. I am pretty sure that most pattern designers are not rolling in cash. I know I’m definitely not! I work one part-time and two casual jobs just to support myself while I run this little business. I would love to be in a position to pay each and every one of my testers. They have incredible skills that I truly value. But at this stage in the game it would be physically impossible. Does that mean I will never pay my testers? Definitely not. As my business expands (hopefully) and generates more income (hopefully), I would love to be sharing the pie around with the people who have helped me get established in the first place. But at this stage the only way I can compensate my testers is with a free pattern and a whole lot of kindness.
There are many ways we can make someone feel appreciated and respected without a financial transaction taking place. And of course, when there is a way I can return the favour, I jump at it with open arms. I am currently testing the Winslow Culottes for Helen from Helen’s Closet. I was so excited when she put a call out for testers. I realised it was a chance to return the favour after she had tested a pattern for me recently. When Andreia contacted me and asked if I would be a sponsor for this years London Sew Social I was happy to donate free patterns for all the attendees, as a way to say thank you to her. My lovely friend Kate, who I share a studio space with, tested for me and I am currently trying to pay her back with tea and treats! That is obviously not all the people who have tested for me, but I will try and repay the favour whenever there is an opportunity to do so. I hope that by building an ongoing relationship with the generous people who have tested for me, I will one day be able to find a way.
Going forward, and having had these issues swirling around my head all week, I have started thinking that maybe I need to work on being more outward about my desire to return the favour and rather than waiting for an opportunity to repay I should be actually asking my testers what I can do in return, what they like in exchange for their time and feedback? So that's some food for thought.
I think that it is very important, that as a designer going into the testing process, we remember that testers are doing us a huge favour, and it is not the other way around. I know that without the collective community that are willing to put their hands up to test a pattern, so many patterns just would not exist. The incredible and diverse range of patterns that we are now lucky enough to have choice of would just not exist without this system. As a designer, I feel completely indebted to all the women who have been involved in testing for me so far, and who test for the independent pattern community in general. If you are a designer and are going into the testing period, go in with this mindset.
