Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

imhashir
Messages : 15
Inscription : lun. juin 17, 2019 12:29 pm

exclamation Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par imhashir » jeu. juil. 11, 2019 2:11 pm

Hi!

I followed the steps of setting callback URL on SquashTM side, installing Jenkins4Sqaush on jenkins and configuring squash tm link on jenkins.
Everything went perfect. I could list down jenkins automated tests from SquashTM and link them with test cases.
I clicked "Run Automated Test" on test case workspace and the selenium automated test started executing. The browser opened, performed some test and closed and jenkins displayed a SUCCESS message. Here is a screenshot of jenkins success message:

Image

But on the TM side, the progress bar doesn't show success or completion and I have to manually change the status to success after seeing completion on Jenkins console.

Please take a look and help me in fixing this issue.

Thank You.

Avatar de l’utilisateur
bfranchet
Messages : 146
Inscription : jeu. nov. 22, 2012 1:36 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par bfranchet » mar. juil. 16, 2019 2:53 pm

Hello,
  • On Jenkins side in the TM publisher configuration section, what's happen when you click on the "validate" button ? (it only verifies that TM is reachable)
  • Have you double check the credentials (login/password) you give in this TM publisher section ? On Squash TM side you should have defined a user with those credentials and member of the group "Test Automation Server"
Regards
PO Squash TF

imhashir
Messages : 15
Inscription : lun. juin 17, 2019 12:29 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par imhashir » mar. juil. 16, 2019 11:41 pm

1. Yes. I clicked the validate button and it displayed success message. TM is perfectly reachable.

2. Yes, the credentials on TM publisher section are of a Test Automation Server user.

Avatar de l’utilisateur
bfranchet
Messages : 146
Inscription : jeu. nov. 22, 2012 1:36 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par bfranchet » mer. juil. 17, 2019 2:46 pm

Hello,

I don't see any "processing test" in your log :

Code : Tout sélectionner

...
[TM-PLUGIN] : identified server : tm-loc
[TM-PLUGIN] : processing test 'testBasicSalaryWithInValidSalary' -> HTTP/1.1 200 
[TM-PLUGIN] : processing test 'testBasicSalaryWithValidSalary' -> not part of the TA test suite definition for this build, no data sent
[TM-PLUGIN] : job done
...
so it seems that the plugin didn't try to send tests status to TM.

Have you in your jenkins job execution log a test section like this one with tests run :

Code : Tout sélectionner

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.howtoprogram.junit5.BasicSalaryCalculatorTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 s - in com.howtoprogram.junit5.BasicSalaryCalculatorTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
If yes, do you see in your job workspace the junit xml report in "target/surefire-reports" ?

Regards
PO Squash TF

imhashir
Messages : 15
Inscription : lun. juin 17, 2019 12:29 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par imhashir » mer. juil. 17, 2019 9:02 pm

Hey!

Thanks for your reply. Yes there are no such logs saying "processing tests".

I don't have the test section like the one you have shown in your reply. Here are my jenkins project configurations:

Image

Avatar de l’utilisateur
bfranchet
Messages : 146
Inscription : jeu. nov. 22, 2012 1:36 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par bfranchet » jeu. juil. 18, 2019 9:02 am

Hello,

Squahs4jenkins should be used to execute tests from TM outside the Squash TA/TF ecosystem. In other words, in this case you have an automation project which generate report in the xml junit format by itself.

If you're trying to execute tests from TM using TF runner or TA/TF scripting then Squahs4Jenkins is not the right way to do it. The documentation for this use case is here : Regards
PO Squash TF

imhashir
Messages : 15
Inscription : lun. juin 17, 2019 12:29 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par imhashir » jeu. juil. 18, 2019 2:56 pm

I have checked again. I have followed all the steps that you mentioned.

This is the project that I am trying to execute:
https://drive.google.com/open?id=11d_A0 ... doIAJfw39B

It has a selenium test case that executes successfully but the result is not returned to TM.

The test is running on Squash TA server. The Execution server job is linked to TM project. The callback url is set.

Here's the complete output of Jenkins console:

Code : Tout sélectionner

Started by user TM Agent
[EnvInject] - Loading node environment variables.
Building in workspace C:\SquashAndTMs\squashta\execution_home\workspace\Test Selenium App - 1
Parsing POMs
Established TCP socket on 49922
[Test Selenium App - 1] $ "C:\Program Files\Java\jdk1.8.0_211/bin/java" -Xms512m -Xmx512m -XX:PermSize=256m -XX:MaxPermSize=512m -cp C:\SquashAndTMs\squashta\execution_home\plugins\maven-plugin\WEB-INF\lib\maven35-agent-1.12-alpha-1.jar;C:\SquashAndTMs\squashta\apache-maven-3.5.0\boot\plexus-classworlds-2.5.2.jar;C:\SquashAndTMs\squashta\apache-maven-3.5.0/conf/logging jenkins.maven3.agent.Maven35Main C:\SquashAndTMs\squashta\apache-maven-3.5.0 C:\SquashAndTMs\squashta\apache-tomcat-8.5.16\webapps\jenkins\WEB-INF\lib\remoting-3.14.jar C:\SquashAndTMs\squashta\execution_home\plugins\maven-plugin\WEB-INF\lib\maven35-interceptor-1.12-alpha-1.jar C:\SquashAndTMs\squashta\execution_home\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.12-alpha-1.jar 49922
<===[JENKINS REMOTING CAPACITY]===>channel started
Executing Maven:  -B -f C:\SquashAndTMs\squashta\execution_home\workspace\Test Selenium App - 1\pom.xml install
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building my.sqaush.ta.project 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- squash-ta-maven-plugin:1.10.0-RELEASE:run (default) @ my.sqaush.ta.project ---
[INFO] Launching Squash TA Chewie edition.
[WARN] Per maven bug #MNG5787, re-adding filtered artifact URL: file:/C:/Users/ruba_hajhamad/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar
[WARN] Per maven bug #MNG5787, re-adding filtered artifact URL: file:/C:/Users/ruba_hajhamad/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar
[WARN] Applied Maven 3.3.x fix
[INFO] Squash TA : compiling tests...
[WARN] The sortMacroList is deprecated, this noop stub is kept for comptibility only
[INFO] Squash TA : initializing context...
[INFO] Listing sqhashTA engine component packages
[INFO] Loading plugin configuration for: org.squashtest.ta.enginecore
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.commons-component
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.filechecker
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.sahi
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.soapui
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.selenium
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.ftp
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.db
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.ssh
[INFO] Loading plugin configuration for: org.squashtest.ta.plugin.local.process
[INFO] Loading XML bean definitions from Byte array resource [Computed squashTA engine configuration]
[INFO] Refreshing org.springframework.context.support.GenericXmlApplicationContext@3b070c7b: startup date [Thu Jul 18 12:51:19 GMT 2019]; root of context hierarchy
[INFO] JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
[WARN] The endpoint URL is set to its default value: "file://dev/null", so the call back is not activated
[INFO] Squash TA : testing...
[WARN] No key selection regex set, target creator may be unusable!
[WARN] No key selection regex set, target creator may be unusable!
[WARN] TA context initialization : configuration warning : repository 'file:/C:/SquashAndTMs/squashta/execution_home/workspace/Test%20Selenium%20App%20-%201/src/squashTA/repositories/sample-repository.properties' could not be instanciated
[WARN] No key selection regex set, target creator may be unusable!
[WARN] No key selection regex set, target creator may be unusable!
[WARN] TA context initialization : configuration warning : repository 'file:/C:/SquashAndTMs/squashta/execution_home/workspace/Test%20Selenium%20App%20-%201/src/squashTA/repositories/_keep' could not be instanciated
[INFO] Beginning execution of ecosystem tests
[INFO] Beginning execution of test test3.ta
[INFO] Exporting results
[INFO] Cleaning resources
[INFO] Squash TA : build complete.
[INFO] All the files from C:\Users\RUBA_H~1\AppData\Local\Temp\2\Squash_TA were properly deleted.
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ my.sqaush.ta.project ---
[INFO] Installing C:\SquashAndTMs\squashta\execution_home\workspace\Test Selenium App - 1\pom.xml to C:\Users\ruba_hajhamad\.m2\repository\org\suashtest\tutorial2\my.sqaush.ta.project\1.0\my.sqaush.ta.project-1.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 38.766 s
[INFO] Finished at: 2019-07-18T12:51:47Z
[INFO] Final Memory: 40M/479M
[INFO] ------------------------------------------------------------------------
[JENKINS] Archiving C:\SquashAndTMs\squashta\execution_home\workspace\Test Selenium App - 1\pom.xml to org.suashtest.tutorial2/my.sqaush.ta.project/1.0/my.sqaush.ta.project-1.0.pom
channel stopped
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=256m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
[htmlpublisher] Archiving HTML reports...
[htmlpublisher] Archiving at PROJECT level C:\SquashAndTMs\squashta\execution_home\workspace\Test Selenium App - 1\target\squashTA\html-reports to C:\SquashAndTMs\squashta\execution_home\jobs\Test Selenium App - 1\htmlreports\HTML_Report
[TM-PLUGIN] : extracting job information
[TM-PLUGIN] : collecting test result data
[TM-PLUGIN] : TA support is enabled
[TM-PLUGIN] : saving the test list
[TM-PLUGIN] : sending results as Squash TA
[TM-PLUGIN] : identifying server : 'http://x4.6x.x9x.1x3/squash'
[TM-PLUGIN] : identified server : TMLocal
[TM-PLUGIN] : job done
Finished: SUCCESS
Here is the log in Tomcat console on execution of test via SquashTM:

Code : Tout sélectionner

18-Jul-2019 12:51:46.768 WARNING [Computer.threadPoolForRemoting [#10] for Channel to Maven [C:\Program Files\Java\jdk1.8.0_211/bin/java, -Xms512m, -Xmx512m, -XX:PermSize=256m, -XX:MaxPermSize=512m, -cp, C:\SquashAndTMs\squashta\execution_home\plugins\maven-plugin\WEB-INF\lib\maven35-agent-1.12-alpha-1.jar;C:\SquashAndTMs\squashta\apache-maven-3.5.0\boot\plexus-classworlds-2.5.2.jar;C:\SquashAndTMs\squashta\apache-maven-3.5.0/conf/logging, jenkins.maven3.agent.Maven35Main, C:\SquashAndTMs\squashta\apache-maven-3.5.0, C:\SquashAndTMs\squashta\apache-tomcat-8.5.16\webapps\jenkins\WEB-INF\lib\remoting-3.14.jar, C:\SquashAndTMs\squashta\execution_home\plugins\maven-plugin\WEB-INF\lib\maven35-interceptor-1.12-alpha-1.jar, C:\SquashAndTMs\squashta\execution_home\plugins\maven-plugin\WEB-INF\lib\maven3-interceptor-commons-1.12-alpha-1.jar, 49922] id=218] hudson.XmlFile.replaceIfNotAtTopLevel JENKINS-45892: reference to hudson.maven.MavenModuleSet@200d05dc[Test Selenium App - 1] being saved from unexpected C:\SquashAndTMs\squashta\execution_home\jobs\Test Selenium App - 1\builds\49\build.xml
 java.lang.IllegalStateException
        at hudson.XmlFile.replaceIfNotAtTopLevel(XmlFile.java:210)
        at hudson.model.AbstractItem.writeReplace(AbstractItem.java:509)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callWriteReplace(SerializationMethodInvoker.java:89)
        at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:141)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
        at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
        at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
        at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
        at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
        at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
        at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
        at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
        at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
        at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)
        at com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)
        at com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
        at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
        at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:265)
        at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:252)
        at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:224)
        at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)
        at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)
        at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)
        at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
        at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
        at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)
        at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)
        at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
        at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)
        at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)
        at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)
        at hudson.XmlFile.write(XmlFile.java:181)
        at hudson.model.Run.save(Run.java:1920)
        at hudson.maven.MavenModuleSetBuild.notifyModuleBuild(MavenModuleSetBuild.java:599)
        at hudson.maven.MavenBuild$ProxyImpl2.end(MavenBuild.java:618)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at hudson.model.Executor$2.call(Executor.java:939)
        at hudson.util.InterceptingProxy$1.invoke(InterceptingProxy.java:23)
        at com.sun.proxy.$Proxy81.end(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922)
        at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896)
        at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
        at hudson.remoting.UserRequest.perform(UserRequest.java:207)
        at hudson.remoting.UserRequest.perform(UserRequest.java:53)
        at hudson.remoting.Request$2.run(Request.java:358)
        at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
        at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19)
        at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

18-Jul-2019 12:51:47.768 INFO [Executor #0 for master : executing Test Selenium App - 1 #49] hudson.model.Run.execute Test Selenium App - 1 #49 main build action completed: SUCCESS

Avatar de l’utilisateur
bfranchet
Messages : 146
Inscription : jeu. nov. 22, 2012 1:36 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par bfranchet » mar. juil. 23, 2019 2:59 pm

Hello,

First of all you have to update the version of the ta / skf framework you use. In your pom.xml I can see :

Code : Tout sélectionner

<ta.framework.version>1.10.0-RELEASE</ta.framework.version>
1.10.0-RELEASE is a buggy version. So you have to update it (last released version is 1.12.0-RELEASE)


I still see in your jenkins stack :

Code : Tout sélectionner

[TM-PLUGIN] : extracting job information
[TM-PLUGIN] : collecting test result data
[TM-PLUGIN] : TA support is enabled
[TM-PLUGIN] : saving the test list
[TM-PLUGIN] : sending results as Squash TA
[TM-PLUGIN] : identifying server : 'http://x4.6x.x9x.1x3/squash'
[TM-PLUGIN] : identified server : TMLocal
[TM-PLUGIN] : job done
That means you didn't deactivated the tm plugin / squash4jenkins plugin for your job. You have to (see my previous answers for explanations about why you don't need this plugin in your case)

Double check in "<your_jenkins_url>/configfiles/editConfig?id=taLinkConf.properties" that one endpoint definition has :
  • the good credentials
  • and the good url( the same as the one define in your tm configuration file )
Then run your jenkins job whith operation list (other parameters don't matter for the operation list). At the ends of this job execution you should have a "Test_list" publish in your jenkins job. When this list is published, you're now able to associate a TM test case to one of the automated test of this published list on TM side.

Now you should be able to launch an execution from TM and retrieve the result in TM.
PO Squash TF

Avatar de l’utilisateur
bfranchet
Messages : 146
Inscription : jeu. nov. 22, 2012 1:36 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par bfranchet » mar. juil. 23, 2019 3:09 pm

By the way I see another strange things in your jenkins job execution log :

Code : Tout sélectionner

Executing Maven:  -B -f C:\SquashAndTMs\squashta\execution_home\workspace\Test Selenium App - 1\pom.xml install
It seems you launch an "install" goal. It's strange. Indeed, when you launch an excution from TM, in the Squash TA framework / SKF context, the job is suppose to execute a "run" goal and not an "install" goal.

I think you've mixed the classic TA framework / skf approach with Squahs4Jeknins / TM publish approach. My advice would be to create a new clean job.
PO Squash TF

imhashir
Messages : 15
Inscription : lun. juin 17, 2019 12:29 pm

Re: Jenkins Automated test executed successfully from SquashTM but success message is not displayed at Squash

Message par imhashir » mer. juil. 24, 2019 1:21 pm

I disabled the Squash4Jenkins plugin. Issue persists.

Checked the squash url and credentials. All good. Issue persists.

I couldn't find the 1.12 version of TA. If you can, please send me a link to that version.

I guess the issue is here. I am not able to list the test cases of jenkins job while picking up Automated Test case in Automation Workspace. This is what appears:

Image

I am manually putting the test case path like this:

/BrowserTest/src/squashTA/resources/selenium-bundle/java/com/example/tests/UntitledTestCase2

or this:

/BrowserTest/src/squashTA/tests/test3.ta

I guess this is the issue. Please guide me how can I execute "run" job and how can I list the test cases.

Thank You.

Répondre