Update master (#60)

* fix temp push

* update version 5.2.1-SNAPSHOT

* fix test case

* fix jetty version,and fix rest api for dataInfoIds

* fix hashcode test

* fix working to init bug

* fix start task log

* fix Watcher can't get providate data,retry and finally return new

* add data server list api

* add server list api

* remove log

* fix isssue 21

* add query by id function

* fix issue 22

* delay client off process and sync data process to working status

* fix data connet meta error

* fix inject NotifyDataSyncHandler

* fix start log

* add send sub log

* fix subscriber to send log

* bugfix: #27

* bugfix: #27

* feature: Add monitoring logs #29

* feature: Add monitoring logs #29
(1) bugfix CommonResponse
(2) format

* bugfix: During meta startup, leader may not register itself #30

* bugfix: Sometimes receive "Not leader" response from leader in OnStartingFollowing() #31

* temp add

* add renew request

* data snapshot module

* add calculate digest service

* fix word cache clientid

* data renew module

* data renew/expired module

* add renew datuem request

* add WriteDataAcceptor

* session renew/expired module

* 1. bugfix ReNewDatumHandler: getByConnectId -> getOwnByConnectId
2. reactor DatumCache from static to instance

* add blacklist wrapper and filter

* upgrade jraft version to 1.2.5

* blacklist ut

* add clientoff delay time

* bugfix： The timing of snapshot construction is not right

* rename: ReNew -> Renew

* fix blacklist test case

* rename: unpub -> unPub

* add threadSize and queueSize limit

* bugfix: revert SessionRegistry

* fix sub fetch retry all error,and reset datainfoid version

* fix client fast chain breakage data can not be cleaned up”

* (1) remove logback.xml DEBUG level;
(2) dataServerBootstrapConfig rename;
(3) print conf when startup

* update log

* fix update zero version,and fix log

* add clientOffDelayMs default value

* fix clientOffDelayMs

* Task(DatumSnapshot/Pub/UnPub) add retry strategy

* bugfix DataNodeServiceImpl: retryTimes

* (1)cancelDataTaskListener duplicate
(2)bugfix DataNodeServiceImpl and SessionRegistry

* refactor datum version

* add hessian black list

* bugfix: log "retryTimes"

* bugfix DatumLeaseManager:  Consider the situation of connectId lose after data restart； ownConnectId should calculate dynamically

* add jvm blacklist api

* fix file name

* some code optimization

* data：refactor snapshot

* fix jetty version

* bugfix DatumLeaseManager: If in a non-working state, cannot clean up because the renew request cannot be received at this time.

* remove SessionSerialFilterResource

* WriteDataProcessor add TaskEvent log; Cache print task update

* data bugfix: snapshot must notify session

* fix SubscriberPushEmptyTask default implement

* merge new

* fix protect

* 1. When the pub of connectId is 0, no clearance action is triggered.
2. Print map. size regularly
3. Delete the log: "ConnectId (% s) expired, lastRenewTime is% s, pub. size is 0"

* DataNodeExchanger： print but ignore if from renew module, cause renew request is too much

* reduce log of renew

* data bugfix: Data coverage is also allowed when versions are equal. Consistent with session design.

* DatumCache bugfix: Index coverage should be updated after pubMap update

* DatumSnapshotHandler: limit print; do not call dataChangeEventCenter.onChange if no diff

* bugfix unpub npe (pub maybe already clean by DatumLeaseManager)；LIMITED_LIST_SIZE_FOR_PRINT change to 30

* some code refactor

* add code comment

* fix data working to init,and fix empty push version

* consider unpub is isWriteRequest, Reduce Snapshot frequency

* RefreshUpdateTime is at the top, otherwise multiple snapshot can be issued concurrently

* update config: reduce retryTimes, increase delayTime, the purpose is to reduce performance consumption

* put resume() in finally code block, avoid lock leak

* modify renewDatumWheelTaskDelay and datumTimeToLiveSec

* When session receives a connection and generates renew tasks, it randomly delays different times to avoid everyone launching renew at the same time.

* data: add executor for handler
session: bugfix snapshot
session: refactor wheelTimer of renew to add executor

* add get data log

* snapshot and lastUpdateTimestamp: Specific to dataServerIP

* 1. DataServer： RenewDatumHandler must return GenericResponse but not CommonResponse, or else session will class cast exception
2. No need to update timestamp after renew
3. snapshot: Need to specify DataServerIP

* add logs

* 1. dataServer: reduce log of snapshotHandler
2. update logs

* dataServer: renew logic should delay for some time after status is WORKING, cause Data is processed asynchronously after synchronization from other DataServer

* bugfix bean; update log

* ignore renew request log

* fix UT

* fix .travis.yml

* fix version 5.3.0-SNAPSHOT

* fix online notify connect error

* fix push confirm error,and fix datum update version,pub threadpool config,add accesslimit service

* add switch renew and expire

* implement renew enable/disable switch

* fix data client exechange log

* fix datum fetch connect error

* bugfix CacheService: set version zero when first sub and get datum error

* fix clean task for fetch

* bugfix DatumCache: Forget to clean up the index in datumCache.putSnapshot

* fix fetch datum word cache

* fix test case time

* fix test cast

* fix test case

* fix tast case

* fix ut case: StopPushDataSwitchTest

* ut case：renew module

* fix ut case：TempPublisherTest

* bugfix ut case: increase sleep time

* fix ut case：RenewTest

* fix ut case：RenewTest format

* fix pom version

* fix ut case：do not run parallelly

client/all/pom.xml

@@ -6,7 +6,7 @@
 
     <groupId>com.alipay.sofa</groupId>
     <artifactId>registry-client-all</artifactId>
-    <version>5.2.1</version>
+    <version>5.3.0-SNAPSHOT</version>
 
     <name>${project.groupId}:${project.artifactId}</name>
     <url>http://github.com/alipay/sofa-registry</url>

client/api/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>com.alipay.sofa</groupId>
         <artifactId>registry-client-parent</artifactId>
-        <version>5.2.1</version>
+        <version>5.3.0-SNAPSHOT</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>

client/impl/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>com.alipay.sofa</groupId>
         <artifactId>registry-client-parent</artifactId>
-        <version>5.2.1</version>
+        <version>5.3.0-SNAPSHOT</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>

client/impl/src/test/java/com/alipay/sofa/registry/client/provider/DefaultRegistryClientTest.java

@@ -16,6 +16,17 @@
  */
 package com.alipay.sofa.registry.client.provider;
 
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import static org.mockito.Mockito.mock;
+
+import org.junit.After;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import com.alipay.sofa.registry.client.api.ConfigDataObserver;
 import com.alipay.sofa.registry.client.api.Configurator;
 import com.alipay.sofa.registry.client.api.Publisher;
@@ -31,16 +42,6 @@
 import com.alipay.sofa.registry.core.model.PublisherRegister;
 import com.alipay.sofa.registry.core.model.ScopeEnum;
 import com.alipay.sofa.registry.core.model.SubscriberRegister;
-import org.junit.After;
-import org.junit.Test;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
-import static org.junit.Assert.assertTrue;
-import static org.mockito.Mockito.mock;
 
 /**
  * The type Default registry client test.
@@ -89,7 +90,7 @@ public void registerPublisher() throws InterruptedException {
 
         defaultPublisher.republish("republish test");
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
         // register success when republish
         assertTrue(defaultPublisher.isRegistered());
     }
@@ -111,7 +112,7 @@ public void handleData(String dataId, UserData data) {
         assertNotNull(subscriber);
         assertEquals(dataId, subscriber.getDataId());
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         RegisterCache registerCache = registryClient.getRegisterCache();
 
@@ -142,7 +143,7 @@ public void handleData(String dataId, ConfigData configData) {
         assertNotNull(configurator);
         assertEquals(dataId, configurator.getDataId());
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         RegisterCache registerCache = registryClient.getRegisterCache();
 
@@ -164,7 +165,7 @@ public void unregisterSinglePublisherTest() throws InterruptedException {
         int unregisterCount = registryClient.unregister(dataId, null, RegistryType.PUBLISHER);
         assertEquals(1, unregisterCount);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         Publisher temp = registryClient.getRegisterCache().getPublisherByRegistId(
             publisher.getRegistId());
@@ -185,7 +186,7 @@ public void unregisterSingleSubscriberTest() throws InterruptedException {
         int unregisterCount = registryClient.unregister(dataId, null, RegistryType.SUBSCRIBER);
         assertEquals(1, unregisterCount);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         Subscriber temp = registryClient.getRegisterCache().getSubscriberByRegistId(
             subscriber.getRegistId());
@@ -206,7 +207,7 @@ public void unregisterSingleConfiguratorTest() throws InterruptedException {
         int unregisterCount = registryClient.unregister(dataId, null, RegistryType.CONFIGURATOR);
         assertEquals(1, unregisterCount);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         Subscriber temp = registryClient.getRegisterCache().getSubscriberByRegistId(
             configurator.getRegistId());
@@ -236,13 +237,13 @@ public void unregisterMultiTest() throws InterruptedException {
 
         Subscriber subscriber2 = registryClient.register(subscriberRegistration2);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         // 2. unregister publisher
         int unregisterCount = registryClient.unregister(dataId, null, RegistryType.PUBLISHER);
         assertEquals(2, unregisterCount);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         // 3. check publisher register cache
         RegisterCache registerCache = registryClient.getRegisterCache();
@@ -256,7 +257,7 @@ public void unregisterMultiTest() throws InterruptedException {
         unregisterCount = registryClient.unregister(dataId, null, RegistryType.SUBSCRIBER);
         assertEquals(2, unregisterCount);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         // 5. check subscriber register cache
         Subscriber tempSub = registerCache.getSubscriberByRegistId(subscriber1.getRegistId());

client/impl/src/test/java/com/alipay/sofa/registry/client/provider/RegisterOrderTest.java

@@ -16,6 +16,20 @@
  */
 package com.alipay.sofa.registry.client.provider;
 
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import java.util.List;
+
+import org.junit.After;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import com.alipay.sofa.registry.client.api.Publisher;
 import com.alipay.sofa.registry.client.api.Subscriber;
 import com.alipay.sofa.registry.client.api.SubscriberDataObserver;
@@ -26,19 +40,6 @@
 import com.alipay.sofa.registry.core.model.DataBox;
 import com.alipay.sofa.registry.core.model.PublisherRegister;
 import com.alipay.sofa.registry.core.model.SubscriberRegister;
-import org.junit.After;
-import org.junit.Test;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import java.util.List;
-
-import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertFalse;
-import static org.junit.Assert.assertNotNull;
-import static org.junit.Assert.assertNull;
-import static org.junit.Assert.assertTrue;
-import static org.junit.Assert.fail;
 
 /**
  *
@@ -126,7 +127,7 @@ public void publishAndUnregister() throws InterruptedException {
         // step 1
         Publisher publisher = registryClient.register(new PublisherRegistration(dataId), data);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         String registId = publisher.getRegistId();
         PublisherRegister publisherRegister = mockServer.queryPubliser(registId);
@@ -137,7 +138,7 @@ public void publishAndUnregister() throws InterruptedException {
         // step 2
         publisher.unregister();
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         assertNull(mockServer.queryPubliser(registId));
 
@@ -151,7 +152,7 @@ public void publishAndRefused() throws InterruptedException {
         // step 1
         Publisher publisher = registryClient.register(new PublisherRegistration(data), data);
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         String registId = publisher.getRegistId();
         PublisherRegister publisherRegister = mockServer.queryPubliser(registId);
@@ -182,7 +183,7 @@ public void handleData(String dataId, UserData data) {
         Subscriber subscriber = registryClient.register(new SubscriberRegistration(dataId,
             dataObserver));
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         String registId = subscriber.getRegistId();
         SubscriberRegister subscriberRegister = mockServer.querySubscriber(registId);
@@ -191,7 +192,7 @@ public void handleData(String dataId, UserData data) {
         // step 2
         subscriber.unregister();
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         assertNull(mockServer.queryPubliser(registId));
 
@@ -211,7 +212,7 @@ public void handleData(String dataId, UserData data) {
         Subscriber subscriber = registryClient.register(new SubscriberRegistration(
             "subscribeAndRefused", dataObserver));
 
-        Thread.sleep(500L);
+        Thread.sleep(2000L);
 
         String registId = subscriber.getRegistId();
         SubscriberRegister subscriberRegister = mockServer.querySubscriber(registId);

client/log/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>com.alipay.sofa</groupId>
         <artifactId>registry-client-parent</artifactId>
-        <version>5.2.1</version>
+        <version>5.3.0-SNAPSHOT</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>

client/pom.xml

@@ -7,7 +7,7 @@
     <parent>
         <groupId>com.alipay.sofa</groupId>
         <artifactId>registry-parent</artifactId>
-        <version>5.2.1</version>
+        <version>5.3.0-SNAPSHOT</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 

core/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>com.alipay.sofa</groupId>
         <artifactId>registry-parent</artifactId>
-        <version>5.2.1</version>
+        <version>5.3.0-SNAPSHOT</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>

pom.xml

@@ -2,11 +2,11 @@
 <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns="http://maven.apache.org/POM/4.0.0"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
-
     <modelVersion>4.0.0</modelVersion>
+
     <groupId>com.alipay.sofa</groupId>
     <artifactId>registry-parent</artifactId>
-    <version>5.2.1</version>
+    <version>5.3.0-SNAPSHOT</version>
     <packaging>pom</packaging>
 
     <name>${project.groupId}:${project.artifactId}</name>
@@ -307,6 +307,12 @@
                 <version>${metrics.version}</version>
             </dependency>
 
+            <dependency>
+                <groupId>commons-collections</groupId>
+                <artifactId>commons-collections</artifactId>
+                <version>3.2.2</version>
+            </dependency>
+
             <!-- junit -->
             <dependency>
                 <groupId>junit</groupId>
@@ -396,6 +402,9 @@
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-surefire-plugin</artifactId>
                 <version>2.21.0</version>
+                <configuration>
+                    <forkMode>once</forkMode>
+                </configuration>
             </plugin>
             <plugin>
                 <groupId>org.jacoco</groupId>

server/common/model/pom.xml

@@ -5,7 +5,7 @@
     <parent>
         <groupId>com.alipay.sofa</groupId>
         <artifactId>registry-common</artifactId>
-        <version>5.2.1</version>
+        <version>5.3.0-SNAPSHOT</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>

server/common/model/src/main/java/com/alipay/sofa/registry/common/model/CommonResponse.java

@@ -108,4 +108,16 @@ public String getMessage() {
     public void setMessage(String message) {
         this.message = message;
     }
+
+    /**
+     * @see Object#toString()
+     */
+    @Override
+    public String toString() {
+        final StringBuilder sb = new StringBuilder("CommonResponse{");
+        sb.append("success=").append(success);
+        sb.append(", message='").append(message).append('\'');
+        sb.append('}');
+        return sb.toString();
+    }
 }