Are you panicking over Etsy's change to make relevancy search the default? Should you be? Visits can fluctuate widely from day to day for reasons that have nothing to do with Etsy's internal search. To really see what's going on you need to look at only visits coming from Etsy's internal search. Here's a quick way to check if it has helped or hurt you. For this you need Site Search tracking set up on Google Analytics (if you don't go here to see how, but I'm not sure if it will give you site search stats retroactively). You could also do this with Etsy's new shop stats but I prefer Analytics for it's ability to do direct comparison of date ranges.
Log into Analytics and go down to Content->Site Search. Set your date range to August 9, 2011 - Yesterday's date (because we want stats for whole days not partial we don't want to include today's date). Then check the box that says "Compare to past" and it should select an equal number of days right before August 9th then click "Apply".In my case the number of searches on Etsy leading to my items increased by 31.85%
Now I don't think this change has been around long enough to get really meaningful results yet, my increases could just be a fluke or because I've added new items, but it's a start and if I had found I was doing worse I'd be in changing my listings to improve them.