mikeash.com pyblog/dont-use-nsoperationqueue.html commentshttp://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsmikeash.com Recent CommentsThu, 28 Mar 2024 21:54:24 GMTPyRSS2Gen-1.0.0http://blogs.law.harvard.edu/tech/rssJeffrey Frey - 2016-07-13 18:47:27http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsPS, I do not get the bug when I add to the operation queue with waitUntilFinished:NO. Maybe just timing. bed232178fcc710d4c29218a2a79e579Wed, 13 Jul 2016 18:47:27 GMTJeffrey Frey - 2016-07-13 18:41:14http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsMike, Thanks very much for this analysis. I have been pulling my hair out for 2 days and did finally realize I am hitting this bug on an iPad IOS 9.3.2 and on the simulator in El Capitan. <br /> <br />6c067898eb1ff5632e76cd0ea9d2d401Wed, 13 Jul 2016 18:41:14 GMTAdeem Basraa - 2011-05-31 09:03:38http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsgetting this issue on ipad 1 but not on ipad2 :(ecffda51453466aa9e31b931c8cdf5d0Tue, 31 May 2011 09:03:38 GMTJon Lund - 2010-09-25 18:10:38http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI'm getting this on an iPad.dcd161923dcd2e3eb4e8a7128555d7f2Sat, 25 Sep 2010 18:10:38 GMTmikeash - 2009-06-05 05:05:54http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsYes. <br /> <br /><a href="http://www.mikeash.com/?page=pyblog/use-nsoperationqueue.html">http://www.mikeash.com/?page=pyblog/use-nsoperationqueue.html</a> <br />eccd729cb20dc2c1919445a0ca4167e4Fri, 05 Jun 2009 05:05:54 GMTCyrus Najmabadi - 2009-06-05 04:56:02http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsHey Mike, any update on this? <br /> <br />Thanks! <br /> <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-- Cyrus5283151af1a47a1b5699f53de940fbb6Fri, 05 Jun 2009 04:56:02 GMTCyrus Najmabadi - 2009-05-13 12:43:58http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsAwesome! I look forward to the results!6968b32266dfd47df67b9e0f056349baWed, 13 May 2009 12:43:58 GMTmikeash - 2009-05-13 04:47:26http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI just updated, and am testing. Post to follow with my results, whatever they are, when I have them.6c8c1af7c260b610d6ef057043c97a9eWed, 13 May 2009 04:47:26 GMTCyrus Najmabadi - 2009-05-13 04:43:15http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsHey Mike, <br /> <br />10.5.7 is out now. I'd love to hear if this issue was resolved in that fix. <br /> <br />Thanks very much for your time and this great article! <br /> <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-- Cyrusa1bd02b5f452e3eb6a95324abb046bc9Wed, 13 May 2009 04:43:15 GMTmikeash - 2009-03-23 23:34:47http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsYeah, I'm hopeful, but I'm not going to call it won until somebody actually tries and verifies the fix. After all, the <i>first</i> version wasn't supposed to be buggy either.126ab19b298faad966b04791e0a49bfcMon, 23 Mar 2009 23:34:47 GMTDave - 2009-03-23 20:35:17http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsLooks like this will be fixed in 10.5.7. <br /> <br />"Fixed problem with NSOperationQueue and PThreads" <br />&lt;&lt; <a href="http://news.worldofapple.com/archives/2009/03/21/apple-seeds-new-build-of-mac-os-x-1057-seed-notes/">http://news.worldofapple.com/archives/2009/03/21/apple-seeds-new-build-of-mac-os-x-1057-seed-notes/</a> <br />8540261c7f287411fbb49202ba1ad40cMon, 23 Mar 2009 20:35:17 GMTJiao Ye - 2009-03-01 12:56:47http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI ran into this discussion by chance, and i tested the codes that Mike posted at 2009-02-01 21:09:55, then boom! these are all i got: <br /> <br />2009-03-01 12:50:08.726 a.out[755:10b] 50000 operations <br />2009-03-01 12:50:21.473 a.out[755:10b] 100000 operations <br />2009-03-01 12:50:31.658 a.out[755:2803] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[MyOperation start]: receiver has already started or finished' <br />2009-03-01 12:50:31.659 a.out[755:2803] Stack: ( <br />&nbsp;&nbsp;&nbsp;&nbsp;2520822027, <br />&nbsp;&nbsp;&nbsp;&nbsp;2483650107, <br />&nbsp;&nbsp;&nbsp;&nbsp;2520821483, <br />&nbsp;&nbsp;&nbsp;&nbsp;2520821546, <br />&nbsp;&nbsp;&nbsp;&nbsp;2490044589, <br />&nbsp;&nbsp;&nbsp;&nbsp;2490043730, <br />&nbsp;&nbsp;&nbsp;&nbsp;2500748303, <br />&nbsp;&nbsp;&nbsp;&nbsp;2500747970 <br />) <br />Trace/BPT trap <br /> <br />I tested the codes with my imac with 2GHz Intel Core 2 Duo and Mac OS X 10.5.6. I wish my post will be helpful. <br />44d2a7c1611ae2967014ba893353c076Sun, 01 Mar 2009 12:56:47 GMTKevin - 2009-03-01 04:56:49http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsMike, your code from 2009-02-01 21:09:55 doesn't crash on my MacBook Pro. The initial code crashes *some of the time* (which is worse than crashing reliably) when compiled for i386, doesn't seem to crash when compiled for x86_64 or PPC. <br /> <br />Model Name: MacBook Pro 15" <br />Model Identifier: MacBookPro2,2 <br />Processor Name: Intel Core 2 Duo <br />Processor Speed: 2.33 GHz <br />Number Of Processors: 1 <br />Total Number Of Cores: 2 <br />L2 Cache: 4 MB <br />Memory: 4 GB <br />Bus Speed: 667 MHz <br />Boot ROM Version: MBP22.00A5.B07 <br />System Version: Mac OS X 10.5.6 (9G55) <br />Kernel Version: Darwin 9.6.0 <br /> <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 677.22.0) <br />MD5 (Foundation) = 9402bcd843fa783b44879ad65f6f09be1943682af9513458a081b7f1ec253d31Sun, 01 Mar 2009 04:56:49 GMTJerry Krinock - 2009-02-04 07:08:22http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI ran Mike's latest code, in the message dated 2009-02-01 21:09:55 (one operation queue, no garbage collection), for over 24 hours, minus a few sleeping hours since I forgot it was running and got no crashes. It logged over 210 million operations before I gave up. I've got an early 2006 Intel Core 2 Duo Mac Mini, 1.66 GHz, 2 GB RAM. <br /> <br />[Session started at 2009-02-02 15:27:01 -0800.] <br />2009-02-02 15:27:16.516 NSOperationCrash[24649:10b] 50000 operations <br />2009-02-02 15:27:29.030 NSOperationCrash[24649:10b] 100000 operations <br />2009-02-02 15:27:40.879 NSOperationCrash[24649:10b] 150000 operations <br />... <br />...the next day... <br />... <br />2009-02-03 14:52:08.011 NSOperationCrash[24649:10b] 209300000 operations <br />2009-02-03 14:52:26.563 NSOperationCrash[24649:10b] 209350000 operations <br />2009-02-03 14:52:44.770 NSOperationCrash[24649:10b] 209400000 operations <br /> <br />Then I ran Mike's original code, from the top of this page, which creates the multiple operation queues, and it crashed after 1:15 min:sec. <br /> <br />If anyone knows how I might make my Mac Mini more "sensitive", I'll give it a try and post the results. <br />229a648c781d6e3787911c6758fe9b17Wed, 04 Feb 2009 07:08:22 GMTFirefly - 2009-02-02 12:14:12http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#comments@mikeash at 2009-02-01 21:09:55: <br /> <br />If you want to test the command-line version with garbage collection enabled: <br />&nbsp;&nbsp;* invoke objc_startCollectorThread() at the beginning of the main function, <br />&nbsp;&nbsp;* invoke -[NSAutoreleasePool drain] instead of -[NSAutoreleasePool release], <br />&nbsp;&nbsp;* optionally, invoke objc_clear_stack() inside the while loop. <br /> <br />"Garbage Collection Programming Guide &gt; Garbage Collection for Cocoa Essentials &gt; Foundation Tools" <br /><a href="http://developer.apple.com/documentation/Cocoa/Conceptual/GarbageCollection/Articles/gcEssentials.html">http://developer.apple.com/documentation/Cocoa/Conceptual/GarbageCollection/Articles/gcEssentials.html</a> <br /> <br />9e4d7778857a0f982be23c15d501512aMon, 02 Feb 2009 12:14:12 GMTmikeash - 2009-02-02 05:09:55http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsGreat work. It seems that you have conclusively demolished the idea that NSOperationQueue is safe as long as there is only one queue in the application, or indeed in any circumstances whatsoever. <br /> <br />I distilled your example code to its essence, removing the GUI and the dependence on garbage collection, and came up with this: <br /> <br /><code>#import &lt;Foundation/Foundation.h&gt; <br /> <br />@interface MyOperation : NSOperation {} @end <br />@implementation MyOperation <br />- (void)main <br />{ <br />&nbsp;&nbsp;&nbsp;&nbsp;usleep(10000); <br />} <br />@end <br /> <br />int main(int argc, char **argv) <br />{ <br />&nbsp;&nbsp;&nbsp;&nbsp;NSAutoreleasePool *outerPool = [[NSAutoreleasePool alloc] init]; <br />&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;NSOperationQueue *queue = [[NSOperationQueue alloc] init]; <br />&nbsp;&nbsp;&nbsp;&nbsp;unsigned count = 0; <br />&nbsp;&nbsp;&nbsp;&nbsp;while(1) <br />&nbsp;&nbsp;&nbsp;&nbsp;{ <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;NSAutoreleasePool *innerPool = [[NSAutoreleasePool alloc] init]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MyOperation *op = [[MyOperation alloc] init]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[queue addOperation:op]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[op release]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if(++count % 50000 == 0) <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;NSLog(@"%u operations", count); <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[innerPool release]; <br />&nbsp;&nbsp;&nbsp;&nbsp;} <br />&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;[outerPool release]; <br />&nbsp;&nbsp;&nbsp;&nbsp;return 0; <br />}</code> <br /> <br />This crashes reliably on my Mac Pro, often after just a few seconds. Sometimes it takes much longer. But it's not doing <i>any</i> of the "funny" stuff that other people have previously complained about or suggested might be behind the problem. I'm not sure how I missed it before, perhaps because I was stuck with the idea of testing self-repeating operations instead of simply enqueueing them separately, but the result here seems pretty solid.92c5b275246162678df037125f3ff3a7Mon, 02 Feb 2009 05:09:55 GMTJulius Guzy - 2009-02-02 03:04:44http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI've done two things, well three, four actually. First I moved the whole main procedure into an object that runs under NSApplicationMain so I could be sure GC applied to it. The first time I ran it it did not crash even after 36 million iterations. The second time it crashed almost straight away with "[NSInvocationOperation start]: receiver has already started or finished" <br /> <br />Next I removed the array of Task objects and replaced it with just a single task object and thereafter all of the NSInvocationOperation objects were created using a call to that object. As in the previous test I cause a wait for the NSOperationQueue to clear every 1000 NSInvocationOperation objects. It crashes with the "receiver has already started or finished" message after 200,000 iterations. <br /> <br />I changed the queue length to 10 and it ran ok until I stopped it at 18 million iterations. Ditto for a queue length of 100. But for a queue length of 500 I did not get past 2 million iterations (4,000 clearings of the queue) before the dreaded "receiver has already started or finished" message. <br /> <br />Unless due to another error it seems then that the problem can also occur with a single queue. The time before errors is erratic, and sometimes if one is running other programs in the foreground these will hang after a few millions of iterations <br /> <br />Finally I ran the same program but with two queues which I waited to clear after putting 10 operations into each of them. Collapse occurred after 640000 iterations with the "receiver has already started or finished" diagnostic. <br /> <br />To make it possible for others to check these results the sources of these XCode 3.1.2 programs running on Mac Pro Leopard 10.5.6 with Garbage Collection Required can be found at: <br /> <br /><a href="http://animatedpaint.co.uk/cocoa-examples-zip/DontUseArrayOfTestObj.zip">http://animatedpaint.co.uk/cocoa-examples-zip/DontUseArrayOfTestObj.zip</a> <br /> <br /><a href="http://animatedpaint.co.uk/cocoa-examples-zip/NewDontUseOneQueue.zip">http://animatedpaint.co.uk/cocoa-examples-zip/NewDontUseOneQueue.zip</a> <br /> <br /><a href="http://animatedpaint.co.uk/cocoa-examples-zip/NewDontUseTwoQueues.zip">http://animatedpaint.co.uk/cocoa-examples-zip/NewDontUseTwoQueues.zip</a> <br /> <br />Best wishes <br />Juliuse0db78efd6790804487faf91ea7c08e5Mon, 02 Feb 2009 03:04:44 GMTmikeash - 2009-02-01 04:03:14http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsYeah, the Cocoa GC is not good with tight loops spilling out all kinds of temporary objects. You'll want to add a call to the collector every so often, or switch to manual memory management. <br /> <br />Also note that your code isn't going to crash according to what's been discovered about the problem so far. The bug only shows up when there are at least two queues in the program. Of course it would be interesting to try it with just one and see if it still crashes, but if you get your memory usage under control and you still don't crash, try adding a second queue.fb8eeb27568c35d97199a7b595adb0f8Sun, 01 Feb 2009 04:03:14 GMTJulius Guzy - 2009-02-01 03:54:46http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsSorry Mike, it took a while for the penny to drop - the GC will only collect after I've exited from the loop so .... <br />I'll recast the problem and take a look at that asap. <br />Julius4f836ca968d2b78b647273cb841d274dSun, 01 Feb 2009 03:54:46 GMTJulius Guzy - 2009-02-01 03:50:59http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsOh and to answer your question, yes the memory does keep growing. <br />e742ddb2063337e856e760ac4e529199Sun, 01 Feb 2009 03:50:59 GMTJulius Guzy - 2009-02-01 03:49:39http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsYes I think it must be something to do with memory management. I am running with GC required. Is there anything else I need to do? The test object is stored temporarily in an NSMutable Array which is replaced every time I allow the queue to clear. <br />Viz my main looks like this <br /><div class="blogcommentquote"><div class="blogcommentquoteinner"> <br />int main(int argc, char **argv) <br />{ <br />&nbsp;&nbsp;&nbsp;NSMutableArray * myArray = [[NSMutableArray alloc]init]; <br />&nbsp;&nbsp;&nbsp;NSOperationQueue * _queue = [[NSOperationQueue alloc] init]; <br />&nbsp;&nbsp;&nbsp;[_queue setMaxConcurrentOperationCount:NSOperationQueueDefaultMaxConcurrentOperationCount]; <br />&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;NSInteger iterationsPerStep = 100; <br />&nbsp;&nbsp;&nbsp;NSInteger interimIterations = 1000; <br />&nbsp;&nbsp;&nbsp;NSInteger i; <br />&nbsp;&nbsp;&nbsp;for(i=0; i &lt; 10000;i++) { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;NSInteger j; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for(j = 0; j &lt; interimIterations; j++) { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;NSInteger k; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;for(k = 0; k &lt; iterationsPerStep; k++) { <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Tester * myTesterObj = [[Tester alloc]init]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[myArray addObject:myTesterObj]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;NSInvocationOperation *op = [[NSInvocationOperation alloc]initWithTarget:myTesterObj <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;selector:@selector(test) <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;object:nil]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[_queue addOperation:op]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} // end for k <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[_queue waitUntilAllOperationsAreFinished]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;myArray = [[NSMutableArray alloc]init]; <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;} // end for j <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;NSLog(@"iterations so far = %d",(i+1) * iterationsPerStep * interimIterations); <br />&nbsp;&nbsp;&nbsp;} // end for i <br />&nbsp;&nbsp;&nbsp; <br />} <br /></div></div> <br /> <br />The extra iteration loop is just a convenience to facilitate printing the numbers. <br />My understanding is that the GC should remove all those now free floating no longer used arrays? <br />Julius24a4a4ee36cbe1ea648ee359ce07fbe4Sun, 01 Feb 2009 03:49:39 GMTmikeash - 2009-01-31 02:33:50http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsYour malloc errors are almost certainly due to a memory management problem. To double check, run Activity Monitor and watch the process's RPRVT number. Does it grow endlessly? If so then that is your problem. Keep in mind that you need an inner autorelease pool to destroy temporary objects if you're looping on the main thread, and also double-check all of your retains and releases to make sure you aren't outright leaking anything.aabdd25ab726c32ac367d575b27a45fbSat, 31 Jan 2009 02:33:50 GMTJulius Guzy - 2009-01-31 00:50:04http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsMike thanks for the posting. <br /> <br />I run a Mac Pro with 2 x 8 core processors. As a newcomer to NSOperationQueue and Cocoa in general I would have spent days looking for the cause of this problem. <br /> <br />Naturally I had to reproduce it my own way. <br />Significantly I used just one queue from main and found that this crashed also! <br /> <br />My tasks did nothing other than run and return. I stuck a counter on it and caused the queue to clear after it had been loaded by n tasks and waited periodically for the queue to clear, all to no avail. For queue lengths of 1000 sometimes it would crash after 20 thousand iterations sometimes 250 thousand exactly as you said it would. <br /> <br />I then decided to give the tasks something to do, just sum the integers from 1 to 1000 and try some different queue lengths. For queue lengths of 10 it would go to 2 million iterations and then hang. For queues of 100 it would go to 3 million and crash with a malloc "unable to allocate" message. Replacing the summation with a sleep(0.01) it got to almost 3 million iterations before collapsing again with the malloc message. <br /> <br />My feeling is that although these latter cases are collapsing for ostensibly different reasons they are nevertheless related. Is is possible that task memory is not being cleared? <br /> <br />Thanks for the alternative code. <br />Best wishes <br />Juliusceb6790f9d9fc8731e3e0468df7f7048Sat, 31 Jan 2009 00:50:04 GMTmikeash - 2009-01-25 12:04:35http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsHaving the selector call itself is intentional. It's to run the operations indefinitely. Changing this to a stub method, and having another method "pump" operations into the queue does not fix the problem. <br /> <br />The reason your altered test case does not crash is because it does not do anything after the initial 10 operations have executed. Run Activity Monitor and watch your test program's CPU usage sit at zero. Put a loop around the initial invocation pump and watch it crash. <br /> <br />The example does not crash due to an infinite loop. It is <i>very clear</i> that it is crashing due to an uncaught exception thrown by NSOperation. It is much more difficult to discover, but just as clear, that this is due to a race condition in NSOperationQueue's background worker threads. <br /> <br />If you have any evidence that the crash is due to running out of memory or such then please post it, but this sort of baseless guessing game is worse than useless.8cd68c90ba1b78c39c68ba43d8489a48Sun, 25 Jan 2009 12:04:35 GMTDevon - 2009-01-25 04:59:36http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsForgive me if I am missing something but after further looking at this you can see that the example code provided to reproduce this problem results in an infinite loop because of the selector being used is _cmd. That is why this example crashes and will depend on the memory/speed and some other factor of your computer as to how soon it will crash. <br /> <br />1636f45a08d70fb833e8fb79cc8d2139Sun, 25 Jan 2009 04:59:36 GMTDevon - 2009-01-25 04:46:50http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI added the method: <br />- (void)test2 <br />{ <br />&nbsp;&nbsp;&nbsp;&nbsp;NSLog(@"test2"); <br />} <br /> <br />and changed the NSInvocationOperation to: <br />NSInvocationOperation *op = [[NSInvocationOperation alloc] <br />&nbsp;&nbsp;&nbsp;&nbsp;initWithTarget:self selector:@selector(test2) object:nil]; <br /> <br />and no longer get the crash. I did get the crash in less than 10 seconds if I left the selector as _cmd6f8f0c12473b9ad64536827ca9bbf95dSun, 25 Jan 2009 04:46:50 GMTDevon - 2009-01-25 04:31:11http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsIts a bit strange that the selector being called is itself in the following line: <br />NSInvocationOperation *op = [[NSInvocationOperation alloc] <br />&nbsp;&nbsp;&nbsp;&nbsp;initWithTarget:self selector:_cmd object:nil]; <br /> <br />It does not make a difference if using a different selector? <br /> <br />e3fc0abdc8e75df0dc963c9185505614Sun, 25 Jan 2009 04:31:11 GMTdreamcat7 - 2009-01-03 17:48:58http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsCocoaDev.com has no pages yet on NSOperation or NSOperationQueue. But this should really all go on a new wiki there for better exposure. <br /> <br />And perhaps a gzipped xcode project would make it easier for others. Then we could just download and run the test on our respective hardware configuration(s) - :) <br /> <br />450810f4ca836b4db5e68774071747faSat, 03 Jan 2009 17:48:58 GMTmikeash - 2008-12-10 04:23:10http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI did contact DTS and they pretty much blew me off. Very disappointing to be sure.6bfcf4239ce0774af715c40d33b70fb2Wed, 10 Dec 2008 04:23:10 GMTJF - 2008-12-10 02:32:09http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsIf you have are select or premier ADC member maybe fill a DTS incident! <br /> <br />I think this would be the best solution to get apples attention on the bug. <br /> <br />0abeda326f1c15fdd6e5072466c774b6Wed, 10 Dec 2008 02:32:09 GMTmikeash - 2008-12-06 09:22:54http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsJust to be clear, the claim that Apple won't fix it for Leopard is <i>entirely</i> my own personal opinion. But here's what leads me to say this: <br /> <br />- My <i>guess</i> is that this is not something which will be fixed by adding an ampersand or something. It seems to be a somewhat more fundamental infrastructure problem which will require a good deal of work to fix, which reduces the probability of getting it fixed. This is pure speculation. <br /> <br />- It's extremely rare. The program in which I discovered this spawns off several hundred NSOperations per second. (I should say spawned, past tense, as it now spawns several hundred RAOperations per second.) Even with this it took an hour to crash on average. A more typical use of NSOperation, from what I've seen, is to do tasks which take multiple seconds each, which enormously increases the probable time-to-crash. <br /> <br />- I discovered this bug about a month ago and as far as I've been able to tell, I am the first. (Nobody I've talked to has said they were familiar with it, and bug report with Apple is still open, although admittedly Apple sometimes leaves bug reports open for years for no reason at all.) This means that it's not being hit often enough for people to notice. That I discovered it doesn't really change that. If this were some sort of catastrophic data loss bug then it might get prioritized, but if Leopard has survived without a fix until now, it'll keep on surviving. <br /> <br />- That it hasn't been found leads me to believe that Apple's use of this API in frameworks is pretty limited. <br /> <br />- Apple has not displayed any particular hurry to fix bad bugs in the past. For one amusing example, try using NSKeyedArchiver to archive the string @"$null" and watch what you get back out. I haven't really tracked that bug but it has probably been there since 10.3. A run through the release notes will turn up a bunch of bugs that never got fixed on 10.X but were fixed for 10.X+1. A big difference is that those bugs tend to be reliable and easy to reproduce, whereas NSOperationQueue breaks only rarely. I don't know if that makes it more or less likely that Apple will fix it. <br /> <br />Anyway, it's just my opinion from watching Apple work that they probably won't fix this until 10.6, but I surely could be wrong.ef765ef59926aab5df9dabc8a5514a70Sat, 06 Dec 2008 09:22:54 GMTJeff Dutky - 2008-12-06 08:15:59http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsOne question: how is it that Apple won't fix this in 10.5? I mean, some of the discussion here seems to imply that NSOperationQueue is actually used by some Apple frameworks (or, at least, that it <i>might</i> be), which would mean that those frameworks will crash so long as this bug remains unfixed. Apple releases all kinds of <i>other</i> bugfixes for older versions of OS X (at least, they're still releasing updated for 10.4 almost a year after 10.5 shipped, if Software Update is to be believed) so I don't see any reason that they couldn't release a fix for NSOperationQueue to 10.5, even after 10.6 ships. <br /> <br />What am I missing?ecc31c0487613e9f6cb04b104e68d3e8Sat, 06 Dec 2008 08:15:59 GMTmikeash - 2008-12-04 01:15:46http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsMy investigation indicates that the bug is that a single NSOperation gets dequeued and executed twice, so definitely an NSOperationQueue bug. As to why you're not seeing it.... <br /> <br />Your operation may not detect that it's being run twice. I'd recommend adding a counter ivar: <br /> <br /><code>int32_t runCount</code> <br /> <br />And then doing an atomic increment plus assert at the top of your operation code: <br /> <br /><code>assert(OSAtomicIncrement32(&amp;runCount) == 1)</code> <br /> <br />This will either blow up when it runs twice, or prove that it's not happening. <br /> <br />Also, it may be much more rare since you're doing actual work. How long does your operation take to run? I believe that each operation basically has a small chance of triggering the bug. My testing indicates that this chance is something on the order of 0.000005 per operation. (This is a very rough estimate.) If your operations take enough time to execute such that running a couple hundred thousand of them takes longer than you've let it run, that could explain it as well. <br /> <br />It could also be that your code is somehow causing NSOperationQueue to run slightly differently and avoid the crash altogether. <br /> <br />And lastly it's certainly possible that my analysis of the bug is wrong.99d1a9d75278313eddd8b75c9b7ab5c3Thu, 04 Dec 2008 01:15:46 GMTLakshmi Vyas - 2008-12-03 17:27:26http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI have a whole bunch of soon to be released code that relies heavily on NSOperationQueue. I started cursing when I saw this post and then replaced the operation in the above code with one of the operations I use very heavily - (Its a custom operation inheriting from NSOperation). It's been running without a crash for the last 10 minutes(the original code crashed). <br /> <br />So, is it an NSOperationQueue bug or an NSInvocationOperation bug? <br />137865834c13df05004ee0575c58b420Wed, 03 Dec 2008 17:27:26 GMTEd Marczak - 2008-12-03 03:47:28http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsToo bad this didn't affect the iPhone, as that would certainly get it fixed sooner.5fbe9aca0ab3096524dd524dfe30df36Wed, 03 Dec 2008 03:47:28 GMTmikeash - 2008-12-03 03:42:58http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsIn case anyone is coming here later on and wondering about alternatives, the replacement I wrote for Rogue Amoeba has now been open sourced: <br /> <br /><a href="http://www.rogueamoeba.com/utm/2008/12/01/raoperationqueue-an-open-source-replacement-for-nsoperationqueue/">http://www.rogueamoeba.com/utm/2008/12/01/raoperationqueue-an-open-source-replacement-for-nsoperationqueue/</a> <br /> <br />It's not a 100% replacement, but it can be very useful in some circumstances.769c0847c09193cbbebe223ff04e9c35Wed, 03 Dec 2008 03:42:58 GMTDavidPhillipOster - 2008-12-03 02:12:18http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsMike Ash is right. In my tests: <br /> <br />PowerPC dual core is OK. <br /> <br />Using a single NSOprationQueue is OK, but you can't guarantee that there is exactly one, since who knows what Apple's libs do. <br /> <br />Intel multi-chip (more than 2 cores) is easy to crash, as Mike says. <br /> <br />Using addDependency: to specify that Op B should not run until after Op A completes does not help. <br /> <br />Using separate classes for the NSOperations does not help. <br /> <br />b92d44ca751fb753dfa9add55236b3baWed, 03 Dec 2008 02:12:18 GMTmikeash - 2008-12-02 04:29:52http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI posted it to OpenRadar (<a href="http://openradar.appspot.com/6332143">http://openradar.appspot.com/6332143</a>). While I think it may help make more people aware of the problem, I doubt that Apple will care at all. <br /> <br />As for iPhones, I suspect that the bug either doesn't happen on ARM or doesn't happen on single-core machines (or both!) so it's probably safe there.c8a3604fc25a6b81150630c61abd9784Tue, 02 Dec 2008 04:29:52 GMTMatt T. - 2008-12-01 21:38:00http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsAfter some more testing, I was able to get it to crash the iPhone simulator. It takes about 40 seconds for me and it doesn't happen every time. <br /> <br />Still haven't been able to crash the actual iPhone. Hopefully it doesn't affect ARM.9f2bfade0a4b83b521b3e3b44059c32bMon, 01 Dec 2008 21:38:00 GMTMatt Mower - 2008-12-01 20:56:51http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsHi Mike. <br /> <br />I haven't used NSOperationQueue although I've been thinking about it which was why I (a) read your post, and (b) became somewhat depressed at your conclusions. <br /> <br />Anyway I've nothing to add except to ask if you think that also filing the bug on OpenRadar (<a href="http://openradar.appspot.com/">http://openradar.appspot.com/</a>) might help raise it's priority? My apologies if it's already there, I couldn't see it. <br /> <br />Regards, <br /> <br />Matt <br /> <br />b6ceb15faae645bd9e241338fa6a4883Mon, 01 Dec 2008 20:56:51 GMTmikeash - 2008-12-01 08:35:27http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI suspect that the iPhone may be immune, either because of ARM or because it only has one CPU core. <br /> <br />I'm surprised that it didn't crash in the simulator, though. That uses your regular CPU and, as far as I understand it, the regular Mac OS X Foundation libraries. Are you able to produce the crash in a regular Mac OS X program? Not everybody has been able to, even on Intel machines (although more than enough have been able to that it's obvious the bug is real and pretty widespread), so maybe your setup is just somehow immune.f265711c7a075f4bce387bf7d239e1a6Mon, 01 Dec 2008 08:35:27 GMTMatt T. - 2008-12-01 08:24:47http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html#commentsI'm curious if this is also an issue on the iPhone OS since it also support NSOperationQueue. I ran the test both on the Simulator and the actual device and I wasn't able to reproduce this crash. Who knows if it's because the bug is rare or if it's Intel only. <br /> <br />This is also worrisome because there are a lot of "New to Cocoa" iPhone developers and I've seen a few use NSOperationQueue because of it's simplicity.ead1982d0a8468210ac6ca44e16aacfcMon, 01 Dec 2008 08:24:47 GMT