Module com.serenitybdd.serenitymodel
Package net.thucydides.model.domain

Class TestOutcome

java.lang.Object
net.thucydides.model.domain.TestOutcome
public class TestOutcome extends Object
Represents the results of a test (or "scenario") execution. This includes the narrative steps taken during the test, screenshots at each step, the results of each step, and the overall result. A test getscenario can be associated with a user story using the UserStory annotation.

A TestOutcome is stored after a test is executed. When the aggregate reports are generated, the test outcome files are loaded into memory and processed.

Author:
johnsmart

  • Constructor Details

    • TestOutcome

      public TestOutcome(String name)
      The title is immutable once set. For convenience, you can create a test run directly with a title using this constructor.
      Parameters:
      name - The name of the Java method that implements this test.

    • TestOutcome

      public TestOutcome(String name, Class<?> testCase)

    • TestOutcome

      public TestOutcome(String name, Class<?> testCase, EnvironmentVariables environmentVariables)
      Create a test outcome based on a test method in a test class. The requirement type will be derived if possible using the class package.
      Parameters:
      name -
      testCase -

    • TestOutcome

      public TestOutcome(String name, Class<?> testCase, String context, EnvironmentVariables environmentVariables)

    • TestOutcome

      protected TestOutcome(String name, Class<?> testCase, Story userStory)

    • TestOutcome

      protected TestOutcome(String name, Class<?> testCase, Story userStory, EnvironmentVariables environmentVariables)
      A test outcome should relate to a particular test class or user story class.
      Parameters:
      name - The name of the Java method implementing this test, if the test is a JUnit or TestNG test (for example)
      testCase - The test class that contains this test method, if the test is a JUnit or TestNG test
      userStory - If the test is not implemented by a Java class (e.g. an easyb story), we may just use the Story class to represent the story in which the test is implemented.

    • TestOutcome

      protected TestOutcome(ZonedDateTime startTime, String name, Class<?> testCase, Story userStory, EnvironmentVariables environmentVariables)

    • TestOutcome

      protected TestOutcome(ZonedDateTime startTime, long duration, String title, String description, String name, String id, String scenarioId, String methodName, Class<?> testCase, List<TestStep> testSteps, List<String> issues, List<String> additionalIssues, List<CastMember> actors, Set<TestTag> tags, TestTag featureTag, Story userStory, FailureCause testFailureCause, String testFailureClassname, String testFailureMessage, String testFailureSummary, TestResult annotatedResult, DataTable dataTable, Optional<String> qualifier, String driver, boolean manualTest, boolean isManualTestingUpToDate, String lastTested, List<String> testEvidence, String projectKey, EnvironmentVariables environmentVariables, ExternalLink externalLink, String context, String testSource)

  • Method Details

    • inEnvironment

      public static net.thucydides.model.domain.TestOutcome.TestOutcomeWithEnvironmentBuilder inEnvironment(EnvironmentVariables environmentVariables)

    • calculateDynamicFieldValues

      public void calculateDynamicFieldValues()
      Fix the values of synthetic fields for serialization purposes

    • usingIssueTracking

      public TestOutcome usingIssueTracking(IssueTracking issueTracking)

    • setToManual

      public TestOutcome setToManual()

    • withTestSource

      public TestOutcome withTestSource(String testSource)

    • withFlagProvider

      protected TestOutcome withFlagProvider(FlagProvider flagProvider)
      Set the current flag provider; only used for testing purposes.

    • setEnvironmentVariables

      public void setEnvironmentVariables(EnvironmentVariables environmentVariables)

    • getEnvironmentVariables

      public EnvironmentVariables getEnvironmentVariables()

    • copy

      public TestOutcome copy()

    • forTest

      public static TestOutcome forTest(String methodName, Class<?> testCase)
      Create a new test outcome instance for a given test class or user story.
      Parameters:
      methodName - The name of the Java method implementing this test,
      testCase - The JUnit or TestNG test class that contains this test method
      Returns:
      A new TestOutcome object for this test.

    • withQualifier

      public TestOutcome withQualifier(String qualifier)

    • withIssues

      public TestOutcome withIssues(List<String> issues)

    • withTags

      public TestOutcome withTags(Set<TestTag> tags)

    • withStartTime

      public TestOutcome withStartTime(ZonedDateTime startTime)

    • getName

      public String getName()
      Returns:
      The name of the Java method implementing this test, if the test is implemented in Java.

    • forTestInStory

      public static TestOutcome forTestInStory(String testName, Story story)

    • forTestInStory

      public static TestOutcome forTestInStory(String testName, Class<?> testCase, Story story)

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getTitle

      public String getTitle()
      Return the human-readable name for this test. This is derived from the test name for tests using a Java implementation, or can also be defined using the Title annotation.
      Returns:
      the human-readable name for this test.

    • getTitle

      public String getTitle(boolean qualified)

    • getUnqualified

      public TitleBuilder getUnqualified()

    • getQualified

      public TitleBuilder getQualified()

    • setAllStepsTo

      public void setAllStepsTo(TestResult result)

    • addDataFrom

      public void addDataFrom(DataTable newDataTable)

    • clearForcedResult

      public void clearForcedResult()

    • setDriver

      public void setDriver(String driver)

    • getDriver

      public String getDriver()

    • getBrowser

      public String getBrowser()

    • resetFailingStepsCausedBy

      public void resetFailingStepsCausedBy(Class<? extends Throwable> expected)

    • replace

      public TestOutcome.StepReplacer replace(List<TestStep> stepsToMerge)

    • mergeMostRecentSteps

      public void mergeMostRecentSteps(int maxStepsToMerge)

    • updateOverallResults

      public void updateOverallResults()

    • getFailingStep

      public Optional<TestStep> getFailingStep()

    • getId

      public String getId()

    • getScenarioId

      public String getScenarioId()

    • getParentId

      public String getParentId()

    • withId

      public TestOutcome withId(String id)

    • withScenarioId

      public TestOutcome withScenarioId(String scenarioId)

    • updateTopLevelStepResultsTo

      public void updateTopLevelStepResultsTo(TestResult result)

    • getTestFailureSummary

      public String getTestFailureSummary()

    • getFlakyTestFailureCause

      public TestFailureCause getFlakyTestFailureCause()

    • setFlakyTestFailureCause

      public void setFlakyTestFailureCause(TestFailureCause flakyTestFailureCause)

    • hasTagWithName

      public boolean hasTagWithName(String tagName)

    • hasTagWithType

      public boolean hasTagWithType(String tagType)

    • hasTagWithTypes

      public boolean hasTagWithTypes(List<String> tagTypes)

    • getDataTableRowCount

      public int getDataTableRowCount()

    • getTestStepCount

      public int getTestStepCount()

    • castActor

      public void castActor(String name)

    • assignFact

      public void assignFact(String name, String fact)

    • assignAbility

      public void assignAbility(String name, String ability)

    • assignDescriptionToActor

      public void assignDescriptionToActor(String name, String description)

    • setManualTestEvidence

      public void setManualTestEvidence(List<String> manualTestEvidence)

    • getManualTestEvidence

      public List<String> getManualTestEvidence()

    • getRenderedManualTestEvidence

      public List<ManualTestEvidence> getRenderedManualTestEvidence()

    • setLink

      public void setLink(ExternalLink externalLink)

    • hasNoSteps

      public boolean hasNoSteps()

    • withSteps

      public TestOutcome withSteps(List<TestStep> childSteps)

    • isUnsuccessful

      public boolean isUnsuccessful()

    • isAJUnit5Test

      public boolean isAJUnit5Test()

    • updateDataTableResult

      public void updateDataTableResult(int row, TestResult result)

    • withTestMethodName

      public TestOutcome withTestMethodName(String methodName)

    • getStepResult

      public TestResult getStepResult(int stepNumber)

    • isTitleWithIssues

      public boolean isTitleWithIssues()

    • setDescription

      public void setDescription(String description)

    • setBackgroundDescription

      public void setBackgroundDescription(String description)

    • setBackgroundTitle

      public void setBackgroundTitle(String title)

    • getDescription

      public String getDescription()

    • getBackgroundDescription

      public String getBackgroundDescription()

    • getBackgroundTitle

      public String getBackgroundTitle()

    • getDescriptionText

      public Optional<String> getDescriptionText()
      Tests may have a description. This can be defined with the scenarios (e.g. in the .feature files for Cucumber) or defined elsewhere, such as in JIRA for manual tests.

    • toJson

      public String toJson()

    • getTitleWithLinks

      public String getTitleWithLinks()

    • getStoryTitle

      public String getStoryTitle()

    • getPath

      public String getPath()

    • getPathId

      public String getPathId()

    • getReportName

      public String getReportName(ReportType type)

    • getSimpleReportName

      public String getSimpleReportName(ReportType type)

    • getHtmlReport

      public String getHtmlReport()

    • getReportName

      public String getReportName()

    • getScreenshotReportName

      public String getScreenshotReportName()

    • getTestSteps

      public List<TestStep> getTestSteps()
      An acceptance test is made up of a series of steps. Each step is in fact a small test, which follows on from the previous one. The outcome of the acceptance test as a whole depends on the outcome of all of the steps.
      Returns:
      A list of top-level test steps for this test.

    • getLatestTopLevelTestStep

      public Optional<TestStep> getLatestTopLevelTestStep()

    • hasScreenshots

      public boolean hasScreenshots()

    • hasRestQueries

      public boolean hasRestQueries()

    • getScreenshotAndHtmlSources

      public List<ScreenshotAndHtmlSource> getScreenshotAndHtmlSources()

    • getScreenshots

      public List<Screenshot> getScreenshots()

    • getStepScreenshots

      public List<Screenshot> getStepScreenshots()
      Find the first and last screenshots for each aggregate step, and every screenshots for leaf steps.

    • hasNonStepFailure

      public boolean hasNonStepFailure()

    • getFlattenedTestSteps

      public List<TestStep> getFlattenedTestSteps()

    • getLeafTestSteps

      public List<TestStep> getLeafTestSteps()

    • getResult

      public TestResult getResult()
      The outcome of the acceptance test, based on the outcome of the test steps. If any steps fail, the test as a whole is considered a failure. If any steps are pending, the test as a whole is considered pending. If all of the steps are ignored, the test will be considered 'ignored'. If all of the tests succeed except the ignored tests, the test is a success. The test result can also be overridden using the 'setResult()' method.
      Returns:
      The outcome of this test.

    • recordSteps

      public TestOutcome recordSteps(List<TestStep> steps)

    • recordChildSteps

      public TestOutcome recordChildSteps(List<TestStep> steps)

    • recordStep

      public TestOutcome recordStep(TestStep step)
      Add a test step to this acceptance test.
      Parameters:
      step - a completed step to be added to this test outcome.
      Returns:
      this TestOucome insstance - this is a convenience to allow method chaining.

    • getFeature

      public ApplicationFeature getFeature()
      Get the feature that includes the user story tested by this test. If no user story is defined, no feature can be returned, so the method returns null. If a user story has been defined without a class (for example, one that has been reloaded), the feature will be built using the feature name and id in the user story.
      Returns:
      The Feature defined for this TestOutcome, if any

    • setTitle

      public void setTitle(String title)

    • startGroup

      @Deprecated public void startGroup(String groupName)
      Deprecated.
      Creates a new step with this name and immediately turns it into a step group.

    • getQualifier

      public Optional<String> getQualifier()

    • startGroup

      public void startGroup()
      Turns the current step into a group. Subsequent steps will be added as children of the current step.

    • endGroup

      public void endGroup()
      Finish the current group. Subsequent steps will be added after the current step.

    • currentStep

      public Optional<TestStep> currentStep()
      Returns:
      The current step is the last step in the step list, or the last step in the children of the current step group.

    • lastStep

      public TestStep lastStep()

    • currentGroup

      public TestStep currentGroup()

    • setUserStory

      public void setUserStory(Story story)

    • determineTestFailureCause

      public void determineTestFailureCause(Throwable cause)

    • appendTestFailure

      public void appendTestFailure(TestFailureCause failureCause)

    • testStepWithDescription

      public Optional<TestStep> testStepWithDescription(String expectedDescription)

    • getTestFailureCause

      public FailureCause getTestFailureCause()

    • getTestFailureErrorType

      public String getTestFailureErrorType()

    • getNestedTestFailureCause

      public FailureCause getNestedTestFailureCause()

    • firstStepWithErrorMessage

      public Optional<TestStep> firstStepWithErrorMessage()

    • testFailureMessage

      public Optional<String> testFailureMessage()

    • getErrorMessage

      public String getErrorMessage()

    • getConciseErrorMessage

      public String getConciseErrorMessage()

    • setTestFailureMessage

      public void setTestFailureMessage(String testFailureMessage)

    • getTestFailureMessage

      public String getTestFailureMessage()

    • getTestFailureClassname

      public String getTestFailureClassname()

    • setAnnotatedResult

      public void setAnnotatedResult(TestResult annotatedResult)

    • overrideAnnotatedResult

      public void overrideAnnotatedResult(TestResult annotatedResult)

    • setResult

      public void setResult(TestResult annotatedResult)

    • overrideResult

      public void overrideResult(TestResult result)

    • withResult

      public TestOutcome withResult(TestResult annotatedResult)

    • getAnnotatedResult

      public TestResult getAnnotatedResult()

    • getAdditionalVersions

      public List<String> getAdditionalVersions()

    • getAdditionalIssues

      public List<String> getAdditionalIssues()

    • getIssues

      public List<String> getIssues()

    • getVersions

      public List<String> getVersions()

    • getTestCase

      public Class<?> getTestCase()

    • getTestCaseName

      public String getTestCaseName()

    • getMethodName

      public String getMethodName()

    • addVersion

      public TestOutcome addVersion(String version)

    • addVersions

      public TestOutcome addVersions(List<String> versions)

    • forProject

      public TestOutcome forProject(String project)

    • getRule

      public Rule getRule()

    • setRule

      public void setRule(Rule rule)

    • getProject

      public String getProject()

    • inTestRunTimestamped

      public TestOutcome inTestRunTimestamped(ZonedDateTime testRunTimestamp)

    • setTestRunTimestamp

      public void setTestRunTimestamp(ZonedDateTime testRunTimestamp)

    • addIssues

      public void addIssues(List<String> issues)

    • getFormattedIssues

      public String getFormattedIssues()

    • isRelatedToIssue

      public void isRelatedToIssue(String issue)

    • addFailingStepAsSibling

      public void addFailingStepAsSibling(List<TestStep> testStepList, Throwable testFailureCause)

    • lastStepFailedWith

      public void lastStepFailedWith(StepFailure failure)

    • lastStepFailedWith

      public void lastStepFailedWith(Throwable testFailureCause)

    • testFailedWith

      public void testFailedWith(Throwable cause)

    • getTags

      public Set<TestTag> getTags()

    • getAllTags

      public Set<TestTag> getAllTags()

    • addUserStoryFeatureTo

      public void addUserStoryFeatureTo(Set<TestTag> augmentedTags)

    • setTags

      public void setTags(Set<TestTag> tags)

    • addTags

      public void addTags(List<TestTag> tags)

    • addTag

      public void addTag(TestTag tag)

    • getIssueKeys

      public List<String> getIssueKeys()

    • getQualifiedMethodName

      public String getQualifiedMethodName()

    • getQualifiedId

      public String getQualifiedId()

    • getNonNullContext

      public String getNonNullContext()

    • getContext

      public String getContext()

    • setContext

      public void setContext(String context)
      Setting the context
      Parameters:
      context -

    • getCompleteName

      public String getCompleteName()
      Returns the name of the test prefixed by the name of the story.

    • useExamplesFrom

      public void useExamplesFrom(DataTable table)

    • addNewExamplesFrom

      public void addNewExamplesFrom(DataTable table)

    • moveToNextRow

      public void moveToNextRow()

    • updateCurrentRowResult

      public void updateCurrentRowResult(TestResult result)

    • dataIsPredefined

      public boolean dataIsPredefined()

    • addRow

      public void addRow(Map<String,?> data)

    • addRow

      public void addRow(DataTableRow dataTableRow)

    • getTestCount

      public int getTestCount()

    • getImplementedTestCount

      public int getImplementedTestCount()

    • countResults

      public int countResults(TestResult expectedResult)

    • countResults

      public int countResults(TestResult expectedResult, TestType expectedType)

    • typeCompatibleWith

      public boolean typeCompatibleWith(TestType testType)

    • countNestedStepsWithResult

      public int countNestedStepsWithResult(TestResult expectedResult, TestType testType)

    • getTagValue

      public Optional<String> getTagValue(String tagType)

    • hasIssue

      public boolean hasIssue(String issue)

    • hasTag

      public boolean hasTag(TestTag tag)

    • hasAMoreGeneralFormOfTag

      public boolean hasAMoreGeneralFormOfTag(TestTag specificTag)

    • hasAMoreSpecificFormOfTag

      public boolean hasAMoreSpecificFormOfTag(TestTag generalTag)

    • setStartTime

      public void setStartTime(ZonedDateTime startTime)

    • clearStartTime

      public void clearStartTime()

    • isManual

      public boolean isManual()

    • getLastTested

      public String getLastTested()

    • setLastTested

      public void setLastTested(String lastTested)

    • isManualTestingUpToDate

      public boolean isManualTestingUpToDate()

    • setManualTestingUpToDate

      public void setManualTestingUpToDate(Boolean upToDate)

    • getFlags

      public Set<? extends Flag> getFlags()

    • isStartTimeNotDefined

      public boolean isStartTimeNotDefined()

    • has

      public TestOutcome.OptionalElements has()

    • getStepCount

      public Integer getStepCount()

    • getRunningStepCount

      public Integer getRunningStepCount()

    • getNestedStepCount

      public Integer getNestedStepCount()

    • getSuccessCount

      public Long getSuccessCount()

    • getFailureCount

      public Long getFailureCount()

    • getErrorCount

      public Long getErrorCount()

    • getCompromisedCount

      public Long getCompromisedCount()

    • getIgnoredCount

      public Long getIgnoredCount()

    • getSkippedOrIgnoredCount

      public Long getSkippedOrIgnoredCount()

    • getSkippedCount

      public Long getSkippedCount()

    • getPendingCount

      public Long getPendingCount()

    • isSuccess

      public Boolean isSuccess()

    • isFailure

      public Boolean isFailure()

    • isCompromised

      public Boolean isCompromised()

    • isError

      public Boolean isError()

    • isPending

      public Boolean isPending()

    • isSkipped

      public Boolean isSkipped()

    • getUserStory

      public Story getUserStory()

    • recordDuration

      public void recordDuration()

    • recordDuration

      public void recordDuration(ZonedDateTime finishTime)

    • setDuration

      public void setDuration(long duration)

    • getDuration

      public Long getDuration()

    • getEndTime

      public ZonedDateTime getEndTime()

    • getDurationInSeconds

      public double getDurationInSeconds()
      Returns:
      The total duration of all of the tests in this set in milliseconds.

    • getVideoLink

      public String getVideoLink()
      Returns the link to the associated video (e.g. from Saucelabs) for this test.
      Returns:
      a URL.

    • getSessionId

      public String getSessionId()

    • setSessionId

      public void setSessionId(String sessionId)

    • countTestSteps

      public Integer countTestSteps()

    • getStartTime

      public ZonedDateTime getStartTime()

    • getStartedAt

      public String getStartedAt()

    • getTimestamp

      public String getTimestamp()

    • getTimestamp

      public String getTimestamp(DateTimeFormatter formater)

    • isDataDriven

      public boolean isDataDriven()

    • getExampleFields

      public List<String> getExampleFields()

    • useScenarioOutline

      public void useScenarioOutline(String scenarioOutline)

    • getDataDrivenSampleScenario

      public String getDataDrivenSampleScenario()

    • getDataTable

      public DataTable getDataTable()

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • getFeatureTag

      public Optional<TestTag> getFeatureTag()

    • getFailureDetails

      public FailureDetails getFailureDetails()

    • getTestSource

      public String getTestSource()

    • setTestSource

      public void setTestSource(String testSource)

    • getActors

      public List<CastMember> getActors()

    • hasEvidence

      public boolean hasEvidence()

    • getEvidence

      public List<ReportData> getEvidence()

    • withDataRowsfilteredbyTag

      public TestOutcome withDataRowsfilteredbyTag(TestTag tag)

    • withDataRowsfilteredbyTagsFrom

      public TestOutcome withDataRowsfilteredbyTagsFrom(Collection<TestTag> filterTags)

    • getExternalLink

      public ExternalLink getExternalLink()

    • setOrder

      public void setOrder(int order)

    • getOrder

      public Integer getOrder()

    • fromStep

      public TestOutcome fromStep(int index)

    • fromStep

      public TestOutcome fromStep(TestStep testStep)

    • withExamplesMatching

      public TestOutcome withExamplesMatching(Predicate<TestStep> condition)

    • withExamplesHavingResult

      public TestOutcome withExamplesHavingResult(TestResult result)

    • asTestCases

      public List<TestOutcome> asTestCases()

    • getTestData

      public String getTestData()

    • removeTopLevelStepsNotMatching

      public TestOutcome removeTopLevelStepsNotMatching(Predicate<TestStep> condition)

    • removeTopLevelStepsNotHavingResult

      public TestOutcome removeTopLevelStepsNotHavingResult(TestResult result)

    • containsAtLeastOneOutcomeWithResult

      public boolean containsAtLeastOneOutcomeWithResult(TestResult expectedResult)

    • getAllResultsStream

      public Stream<TestResult> getAllResultsStream()

    • getAllResults

      public List<TestResult> getAllResults()

    • getResultCount

      public long getResultCount()

    • getDistinctResults

      public Set<TestResult> getDistinctResults()

    • hasResult

      public boolean hasResult(TestResult result)