o >h@ @s2dZddlZddlZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZmZddlmZmZddlmZmZmZddlmZdd lmZmZmZdd lm Z m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6m7Z7ddl8m9Z9m:Z:ddl;mZ>m?Z?m@Z@mAZAddlBmCZCmDZDmEZEGddde e!ZFdeFfddZGeGd d!d!ZHeGd"d#d#ZIeGd$d%d%ZJd&eKd'e/deKfd(d)ZLd*eId+e e0d'es  z(StartedWorkerPool.run..N)rrM)selfrNr6rSr7runs  zStartedWorkerPool.runcstdd|jDddIdH}t|D]\}\}}|s'|jd|d|q|jdd=|jdd=|j|j dS)zv Shut down all of the workers in the pool. The pool is unusable after this method is called. cSsg|] }t|qSr6)r fromCoroutineexitrPr6r6r7 sz*StartedWorkerPool.join..T) consumeErrorsNzjoining disttrial worker #z failed) rrL enumerate_loggerfailurerMrKcloserJunlock)rUresultsn succeededr]r6r6r7joins    zStartedWorkerPool.join)r8N)r2r3r4r5rrrGrr rr.r/rr\r0rVrcr6r6r6r7rIcs     rIc @sZeZdZUdZeed<deedee de de e fddZ d d Zd edefd d ZdS) WorkerPoolz Manage a fixed-size collection of child processes which can run tests. @ivar _config: Configuration for the precise way in which the pool is run. _config protocolsrCrEr8csfddt|DS)a Create local worker protocol instances and return them. @param protocols: The process/protocol adapters to use for the created workers. @param workingDirectory: The base path in which we should run the workers. @param logFile: The test log, for workers to write to. @return: A list of C{quantity} C{LocalWorker} instances. cs&g|]\}}t|t|qSr6)r.childrH)rQxprotocolrErCr6r7rYsz2WorkerPool._createLocalWorkers..)r[)rUrfrCrEr6rjr7_createLocalWorkerss zWorkerPool._createLocalWorkersc Csvtdjj}ddddddtdtdi}tj}tj t j|d<|D]}t j |g}| |||t j |||dq"d S) a Spawn processes from a list of process protocols. @param spawner: A C{IReactorProcess.spawnProcess} implementation. @param protocols: An iterable of C{ProcessProtocol} instances. @param arguments: Extra arguments passed to the processes. ztwisted.trial._dist.workertrialrwr&rr PYTHONPATH)argschildFDsenvN) rfilePathpathr'r(osenvironcopypathseprcsys executableextend) rUspawnerrf argumentsworkertrialPathrprurRror6r6r7_launchWorkerProcessess    z!WorkerPool._launchWorkerProcessesr;cst|jj\}}t|jjrt|jj}n||jj}t|}ddt|jj D}| |||}| |j ||jj t|||||S)z Launch all of the workers for this pool. @return: A started pool object that can run jobs using the workers. cSsg|]}tqSr6)r/)rQrhr6r6r7rYsz$WorkerPool.start..)r$rerCrrEr preauthChildr%rangerBrkr~ spawnProcessrDrI)rUr;testDirrJ testLogPathrKrMrLr6r6r7starts4 zWorkerPool.startN)r2r3r4r5rArGrr/rrr rr.rkr~rrIrr6r6r6r7rds  rd untilFailureresultcCs |o|S)z Determine whether the test suite should be iterated again. @param untilFailure: C{True} if the suite is supposed to run until failure. @param result: The test result of the test suite iteration which just completed. ) wasSuccessful)rrr6r6r7shouldContinues rpool testCases driveWorkercsJz|t|||IdHWdSty$|jtdtYdSw)Nz )rVr ExceptionoriginaladdErrorr#r)rrrrr6r6r7runTestss  rc@szeZdZUdZeZeZede fe d<e e d<e e e d<dZee d<edeed d Zee d <edeejd Zee d <d Ze e d<dZee d<dZee d<dZe e d<dZe e d<eZee gefe d<defddZ!ddZ"dede#e$de%ddfddZ& d+d e'e(e)fd!edefd"d#Z*d$e'e(e)fd!ede fd%d&Z+d$e'e(e)fde fd'd(Z,d$e'e(e)fde fd)d*Z-dS),DistTrialRunnera A specialized runner for distributed trial. The runner launches a number of local worker processes which will run tests. @ivar _maxWorkers: the number of workers to be spawned. @ivar _exitFirst: ``True`` to stop the run as soon as a test case fails. ``False`` to run through the whole suite and report all of the results at the end. @ivar stream: stream which the reporter will use. @ivar _reporterFactory: the reporter class to be used. ._reporterFactory _maxWorkers_workerArgumentsF _exitFirstN)factory)default converter_reactorstreamr_tracebackFormat_realTimeErrors_uncleanWarningsztest.log_logfile _trial_temp_workingDirectory_workerPoolFactoryr8cCs.|j|j|j|jd}|jrt|}||S)zL Make reporter factory, and wrap it with a L{DistReporter}. )realtime)rrrrrr"_distReporterFactory)rUreporterr6r6r7 _makeResultJs   zDistTrialRunner._makeResultcCs |dS)z Write test run final outcome to result. @param result: A C{TestResult} which will print errors and the summary. N)done)rUrr6r6r7 writeResultsUs zDistTrialRunner.writeResultsrrrRcs,fdd}|D] }||IdHq dS)a Drive a L{LocalWorkerAMP} instance, iterating the tests and calling C{run} for every one of them. @param worker: The L{LocalWorkerAMP} to drive. @param result: The global L{DistReporter} instance. @param testCases: The global list of tests to iterate. @return: A coroutine that completes after all of the tests have completed. cs@z |IdHWdStyj|tYdSwrO)rVrrrr)caserrRr6r7taskqs  z*DistTrialRunner._driveWorker..taskNr6)rUrrrRrrr6rr7 _driveWorker]s zDistTrialRunner._driveWorkersuitercstt|tttjtjj j }j d| d|jIdHtt}tdtdtffdd }zt||IdHWIdHSIdHw)a* Spawn local worker processes and load tests. After that, run them. @param suite: A test or suite to be run. @param untilFailure: If C{True}, continue to run the tests until they fail. @return: A coroutine that completes with the test result. zRunning z tests. Nrar8cslrjd|ddjrfdd}ndd}tt|jIdHS)Nz Test Pass r& cs jSrO)rr_rr6r7s z@DistTrialRunner.runAsync..runAndReport..cSsdSNTr6rr6r6r7rs)rwriterrrr-rr)racasesConditionrU startedPoolrrrr7 runAndReports z.DistTrialRunner.runAsync..runAndReport)listrrrAminlenrrrrrrrcountTestCasesrrrrr*rFr)r,rc)rUrr poolStarter conditionrr6rr7runAsynczs$    zDistTrialRunner.runAsynctestcsdddtttfddffdd }dtttffdd }dtdtffd d }jd d |t || | |j t trS t ts_Jd ttjS)NFrmr8cs|dSrOr6)rmrr6r7capturesz%DistTrialRunner._run..capturecsddur SdSr)cancelr6)reactorStoppingrtestsInProgressr6r7maybeStopTestss z,DistTrialRunner._run..maybeStopTestsrcssj|SrO)rstopr)rrUr6r7maybeStopReactors z.DistTrialRunner._run..maybeStopReactorbeforeshutdownz is not DistReporter)r rr)r robjectraddSystemEventTriggerrWraddBothrV isinstanceraiseExceptionr r r)rUrrrrrr6)rrrUrr7_runs   zDistTrialRunner._runcC|j|ddS)za Run a reactor and a test suite. @param test: The test or suite to run. FrrrUrr6r6r7rVzDistTrialRunner.runcCr)z| Run the tests with local worker processes until they fail. @param test: The test or suite to run. Trrrr6r6r7runUntilFailurerzDistTrialRunner.runUntilFailure)F).r2r3r4r5r)rrr\rr rGrFrrHrboolrrr@rr1rxstdoutrr rrrrrrdrrArrr r!r/rr rrrrrVrr6r6r6r7r!sR             L,r)Or5rtrx functoolsros.pathrtypingrrrrrr r r r r unittestrrattrsrrrattrs.convertersrtwisted.internet.deferrrrtwisted.internet.interfacesrrtwisted.loggerrtwisted.python.failurertwisted.python.filepathrtwisted.python.lockfilertwisted.python.modulesr _asyncrunnerritrialr r!rr"runnerr#utilr$r%r'r( distreporterr) functionalr*r+r,r-rRr.r/r0r1r@rArIrdrrrrr6r6r6r7sb  0           8e