Elasticsearch provide some test facilities officially. ESIntegTestCase allow you to start a local elasticsearch cluster from test container, so that you can test elasticsearch index/search/aggregation without mocking. However, there are actually quite a few pitfalls in order to make it work.

為元謀等地區用戶提供了全套網頁設計制作服務，及元謀網站建設行業解決方案。主營業務為成都網站建設、成都網站制作、元謀網站設計，以傳統方式定制建設網站，并提供域名空間備案等一條龍服務，秉承以專業、用心的態度為用戶提供真誠的服務。我們深信只要達到每一位用戶的要求，就會得到認可，從而選擇與我們長期合作。這樣，我們也可以走得更遠！

Basic setup:
Gradle dependencies:

testCompile 'org.elasticsearch.test:framework:6.5.2'
testCompile 'org.elasticsearch.plugin:transport-netty4-client:6.5.2'
testCompile 'org.apache.logging.log4j:log4j-slf4j-impl:2.9.0'

Also in order to use java 8 features in kotlin in integration test cases, the java version has to be declared explicitly. Note in gradle, you need separation declaration for class compilation and test class compilation，which is different from maven

compileTestKotlin {
        sourceCompatibility = JavaVersion.VERSION_1_8
        targetCompatibility = JavaVersion.VERSION_1_8

        kotlinOptions {
            jvmTarget = "1.8"
            apiVersion = "1.2"
            languageVersion = "1.2"
        }
    }

Setup test case:
The test case needs to inherit from ESIntegTestCase. It automatically starts an elasticsearch cluster in "beforeClass" or "before" method depending on the cluster scope (suite or test). @ESIntegTestCaseannotation specifies how cluster is setup, in this case, we need only a single data node. Without specifying the node settings, ESIntegTestCase might start a cluster which random number of cluster nodes. @ThreadLeakScopeannotation prevents error messages printed on console when elasticsearch server performs thread leak check. Once the test case is set up, it should be able to use "client()" method to get a elasticsearch client instance for various operation in test case.

@ESIntegTestCase.ClusterScope(numDataNodes=1, numClientNodes=0)
@ThreadLeakScope(ThreadLeakScope.Scope.NONE)
public class RuleIntegratedTest : ESIntegTestCase()

Support remote connection:
We are using the new high level rest client in favor of the deprecated transport client. However, ESIntegTestCase does not expose a remote port for http connection. It only starts a local mock transport. To resolve this issue, we need to customize the node building by add a netty4 transport plugin. This can be achieved by overriding "nodeSettings" and "nodePlugins" method

override fun nodeSettings(nodeOrdinal : Int) : Settings {
    val randomPort = Random().ints(1, 15000, 20000).findFirst().asInt
    ruleLogger.debug("Random port is {}", randomPort)
    return Settings.builder().put(super.nodeSettings(nodeOrdinal))
            .put(NetworkModule.HTTP_ENABLED.key, true)
            .put(NetworkModule.HTTP_TYPE_KEY, "netty4")
            .put(HttpTransportSettings.SETTING_HTTP_PORT.key, randomPort).put("network.host", "127.0.0.1")
            .build()
}

override fun nodePlugins(): MutableCollection<Class<out Plugin>> {
    val result = mutableListOf<Class<out Plugin>>()
    result.add(Netty4Plugin::class.java)
    return result
}

To build the rest client, we need to know the elasticsearch server's port. This can be achieved by exploit the low level "client()" exposed by ESIntegTestCase. It sends a request to server to obtain cluster information, and host/port information can be obtained from there.

fun buildHighLevelClientFromClient(client: Client): RestHighLevelClient {
    val nodesInfoResponse = client.admin().cluster().prepareNodesInfo(*arrayOfNulls(0)).get() as NodesInfoResponse
    val nodes = nodesInfoResponse.nodes
    val httpHosts = ArrayList<HttpHost>()
    for (node in nodes) {
        logger.debug("Finding next node: {}",  node)
        if (node.http != null) {
            val publishAddress = node.http.address().publishAddress()
            val address = publishAddress.address()
            httpHosts.add(HttpHost(NetworkAddress.format(address.address), address.port))
        }
    }

    logger.info("Completed building hosts: {}", httpHosts)

    return RestHighLevelClient(
            RestClient.builder(*httpHosts.toTypedArray()))
}

I was getting a strange netty 4 processor issue when running test in maven

java.lang.IllegalStateException: available processors value [1] did not match current value [3]
        at org.elasticsearch.transport.netty4.Netty4Utils.setAvailableProcessors(Netty4Utils.java:90)
        at org.elasticsearch.http.netty4.Netty4HttpServerTransport.<init>(Netty4HttpServerTransport.java:252)
        at org.elasticsearch.transport.Netty4Plugin.lambda$getHttpTransports$1(Netty4Plugin.java:98)

So a system properties "es.set.netty.runtime.available.processors" has to be set to "false" to resolve this issue. However, i was not getting the same error in gradle, so it is not absolutely necessary.

test {
    testLogging {
        showStandardStreams = true
        events "PASSED", "STARTED", "FAILED", "SKIPPED", "STANDARD_OUT", "STANDARD_ERROR"
    }

    systemProperties = [
        "es.set.netty.runtime.available.processors": "false",
        "tests.security.manager": "false"
    ]
}

After this, some security error emerges when running the test case

[2019-01-26T12:47:14,358][ERROR][i.n.u.c.D.rejectedExecution] [node_sd3] Failed to submit a listener notification task. Event loop shut down?
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "setContextClassLoader")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_191]
        at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_191]
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_191]
        at java.lang.Thread.setContextClassLoader(Thread.java:1474) ~[?:1.8.0_191]
        at io.netty.util.concurrent.GlobalEventExecutor$2.run(GlobalEventExecutor.java:228) ~[netty-common-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.GlobalEventExecutor$2.run(GlobalEventExecutor.java:225) ~[netty-common-4.1.30.Final.jar:4.1.30.Final]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_191]
        at io.netty.util.concurrent.GlobalEventExecutor.startThread(GlobalEventExecutor.java:225) ~[netty-common-4.1.30.Final.jar:4.1.30.Final]

If you notice, in elasticsearch module folder: elasticsearch-6.5.4\modules\transport-netty4, there is a customized java security policy file: plugin-security.policy, which defines customized security right when start netty. I imported it into the test folder and use it when running elasticsearch integration test.

Gradle:

test {
    testLogging {
        showStandardStreams = true
        events "PASSED", "STARTED", "FAILED", "SKIPPED", "STANDARD_OUT", "STANDARD_ERROR"
    }

    systemProperties = [
        "es.set.netty.runtime.available.processors": "false",
        "java.security.policy": "${projectDir}/plugin-security.policy"
    ]
}

Maven:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <systemPropertyVariables>
                    <java.security.policy>${project.basedir}/plugin-security.policy</java.security.policy>
                    <es.set.netty.runtime.available.processors>false</es.set.netty.runtime.available.processors>
                    <buildDirectory>${project.build.directory}</buildDirectory>
                </systemPropertyVariables>
            </configuration>
        </plugin>
    </plugins>   
</build>

For maven, this is enough to make it work normally, But in gradle, importing a policy file is not sufficient to fully resolve the security issue. Gradle forks a separate process to run the test case, and due to the error, the test runner hangs indefinitely and no error will be shown in the console! Even a java thread dump does not show any useful information because the forked process dies and the test runner simply wait forever a response that is never returned. The correct approach is to disable java security policy completely by setting system property "tests.security.manager" = "false"

Jar Hell Issue:
Elasticsearch will perform jar hell check on startup. Because elasticsearch uses a plugin architecture, the jars ported with plugin could potentially cause library conflict. Jar hell check is to check whether conflict jar version occurs in classpath. When starting elasticsearch integration test in gradle, jar hell check fails. It complains about some hamcrest version issue. Googling on web, I found two solutions to this issue. I adopted the second solution.

1. Exclude hamcrest depedencies when compiling test case:

testCompile (group: 'junit', name: 'junit', version: '4.11') {
    exclude group:'org.hamcrest' //also included in es test framework
}

2. Disable jar hell check: create a JarHell check class with empty implementation and replace the default one:

package org.elasticsearch.bootstrap;

import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.Collections;
import java.util.Set;
import java.util.function.Consumer;

public class JarHell {
    private JarHell() {}
    public static void checkJarHell(Consumer<String> output) throws IOException, URISyntaxException {}

    public static Set<URL> parseClassPath() { return Collections.emptySet(); }

    public static void checkJarHell(Set<URL> urls, Consumer<String> output) throws URISyntaxException, IOException {}

    public static void checkVersionFormat(String targetVersion) {}

    public static void checkJavaVersion(String resource, String targetVersion) {}

}

Reference:

https://discuss.elastic.co/t/elasticsearch-5-1-1-simple-integration-test-hangs-running-gradle-test/70485

https://stackoverflow.com/questions/33975807/elasticsearch-jar-hell-error

網頁標題：GetElasticsearchintegrationtestwithgradle
文章轉載：http://vcdvsql.cn/article32/jhjgsc.html

成都網站建設公司_創新互聯，為您提供手機網站建設、App設計、軟件開發、品牌網站制作、網頁設計公司、網站導航

廣告

聲明：本網站發布的內容（圖片、視頻和文字）以用戶投稿、用戶轉載內容為主，如果涉及侵權請盡快告知，我們將會在第一時間刪除。文章觀點不代表本網站立場，如需處理請聯系客服。電話：028-86922220；郵箱：631063699@qq.com。內容未經允許不得轉載，或轉載時需注明來源： 創新互聯