Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης...

22
Dynamic-Content Web Dynamic-Content Web Caching with Caching with Cooperative Proxy Cooperative Proxy Scheme Scheme Βελισκάκης Βελισκάκης Μανώλης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Dept. of Electrical & Computer Engineering Engineering Knowledge and Database Systems Knowledge and Database Systems Laboratory Laboratory Συνάντηση Συνάντηση DBLAB DBLAB Τρίτη, Τρίτη, 20 Ιανουαρίου 2004 20 Ιανουαρίου 2004

Transcript of Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης...

Page 1: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Dynamic-Content Web Caching Dynamic-Content Web Caching withwith

Cooperative Proxy Scheme Cooperative Proxy Scheme

Βελισκάκης ΜανώληςΒελισκάκης ΜανώληςΕθνικό Μετσόβιο ΠολυτεχνείοΕθνικό Μετσόβιο Πολυτεχνείο

Dept. of Electrical & Computer EngineeringDept. of Electrical & Computer Engineering

Knowledge and Database Systems LaboratoryKnowledge and Database Systems Laboratory

Συνάντηση Συνάντηση DBLABDBLABΤρίτη, Τρίτη, 20 Ιανουαρίου 200420 Ιανουαρίου 2004

Page 2: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

OutlineOutline

Problem DefinitionProblem Definition Dynamic-Data Web Caching vs Dynamic-Data Web Caching vs

Cooperative SchemesCooperative Schemes Proposed Web Caching AlgorithmProposed Web Caching Algorithm Current and Future WorkCurrent and Future Work DiscussionDiscussion

Page 3: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Problem Definition – What?Problem Definition – What?

Query Results

Dynamic Data for personalization purposes

Page 4: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Problem Definition – Where?Problem Definition – Where?

ClientClient ProxyProxy Edge-of-netEdge-of-net Internet Service ProviderInternet Service Provider Edge-of-EnterpriseEdge-of-Enterprise Application ServerApplication Server Web ServerWeb Server DBMSDBMS

Page 5: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Problem Definition – How?Problem Definition – How?

Nowadays ApproachesNowadays Approaches

Exact matching queryExact matching query Materialized ViewsMaterialized Views DB Characteristics to ProxiesDB Characteristics to Proxies

Page 6: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Problem Definition – Topology Problem Definition – Topology SchemeScheme

Broadcast queriesBroadcast queries Hierarchical CachingHierarchical Caching URL HashingURL Hashing Directory based CooperationDirectory based Cooperation

Page 7: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Problem Definition - IssuesProblem Definition - Issues

Replacement PolicyReplacement Policy Cache ConsistencyCache Consistency Proxy CommunicationProxy Communication Web objects placementWeb objects placement

Page 8: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Dynamic-Data Web Caching Dynamic-Data Web Caching vs Cooperative Schemesvs Cooperative Schemes

Exact matching Exact matching queryquery

Materialized ViewsMaterialized Views DB Characteristics DB Characteristics

to Proxiesto Proxies

Broadcast queriesBroadcast queries Hierarchical Hierarchical

CachingCaching URL HashingURL Hashing Directory based Directory based

CooperationCooperation

Replacement PolicyReplacement Policy

Proxy CommunicationProxy Communication

Web objects placementWeb objects placement

Page 9: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Dynamic-Data Web Caching Dynamic-Data Web Caching vs Cooperative Schemesvs Cooperative Schemes

Conclusions (?)Conclusions (?)

Exact Matching Query Exact Matching Query

– Common Web Caching IssuesCommon Web Caching Issues

– Not interesting Not interesting DB Characteristics to ProxiesDB Characteristics to Proxies

– Common DB Replication IssuesCommon DB Replication Issues

– Interesting Issue: Create Cache Tables knowing that there is a Interesting Issue: Create Cache Tables knowing that there is a cooperative proxy Schemecooperative proxy Scheme

Page 10: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Dynamic-Data Web Caching Dynamic-Data Web Caching vs Cooperative Schemesvs Cooperative Schemes

Conclusions (?)Conclusions (?) Materialized ViewsMaterialized Views

– Many interesting issues Many interesting issues Query rewriting Query rewriting Replacement AlgorithmReplacement AlgorithmAppropriate Cooperative SchemeAppropriate Cooperative SchemeWeb Objects exchange between ProxiesWeb Objects exchange between ProxiesConsideration of DBMS structure Consideration of DBMS structure Dynamic or a priori definition of Materialized ViewsDynamic or a priori definition of Materialized ViewsGiving DB capabilities to Proxies (queries on Materialized Giving DB capabilities to Proxies (queries on Materialized

Views)Views)Communication between ProxiesCommunication between Proxies

Page 11: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Hybrid Topology (Hierarchical-Directory Based)Hybrid Topology (Hierarchical-Directory Based)

PROXY 1b

DIRECTORY

Q.MQ.M

CACHECACHE

PROXY 2b

DIRECTORY

Q.MQ.M

CACHECACHE

PROXY 1c

DIRECTORY

Q.MQ.M

CACHECACHE

PROXY 1aPROXY 1aCCLLIIEENNTTSS

DIRECTORYDIRECTORY

Q.MQ.M

CACHECACHE

PROXY 2c

DIRECTORY

Q.MQ.M

CACHECACHE

PROXY 2aCCLILIEENNTTSS

DIRECTORY

Q.MQ.M

CACHECACHE

WEB SERVERWEB SERVER

DATABASEDATABASESERVERSERVER

WEB SERVERWEB SERVER

DATABASESERVER

Page 12: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Web Objects descriptionWeb Objects description

There are 3 different ways to refer to a There are 3 different ways to refer to a Web ObjectWeb Object– URLURL

– QTagQTag

– QTag+Query Result (Whole Web Object)QTag+Query Result (Whole Web Object)

Page 13: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Web Objects descriptionWeb Objects description

QTAGQTAG

<QTag <QTag ID:Number, ID:Number, //Unique identifier for every QTag//Unique identifier for every QTag Query:String, Query:String, //Contains the query that has been asked to the Back-//Contains the query that has been asked to the Back-End End DatabaseDatabase

LocationOfWebServer:URLLocationOfWebServer:URL, , //Contains the URL Location of the Web Server that //Contains the URL Location of the Web Server that stands stands in front of the Databasein front of the Database

DatabaseID:NumberDatabaseID:Number,, //Contains the ID of the Database where the query was //Contains the ID of the Database where the query was asked asked

TimeToLive:Number (sec), TimeToLive:Number (sec), //Determines the period in which the query is valid and can //Determines the period in which the query is valid and can satisfy Requestssatisfy Requests

Weight:NumberWeight:Number,, //Determines the significance (Weight) of the query. //Determines the significance (Weight) of the query.

Relationships:List of QTag.ID Relationships:List of QTag.ID //Determines a list of Web Objects that are //Determines a list of Web Objects that are frequently used with the current frequently used with the current

Web Object in Web Object in order to satisfy query order to satisfy query requestsrequests

/> />

Page 14: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Web Objects descriptionWeb Objects description

QTAG + Query ResultsQTAG + Query Results

<QTag<QTagID=1029384,ID=1029384,QueryQuery=”Select name, surname, age from Customers where =”Select name, surname, age from Customers where

Name=’John’”, Name=’John’”,LocationOfWebServerLocationOfWebServer =”http:// =”http://www.www.dblabdblab..eceece..ntuantua..grgr/siteNo1/siteNo1”, ”, DatabaseIDDatabaseID

=1,=1,TimeToLiveTimeToLive=1000,=1000,WeightWeight=0.65=0.65Relationships=”15433456, 15433766, 15682456, 15432456Relationships=”15433456, 15433766, 15682456, 15432456

/>/>John, Manolopoulos, 28John, Manolopoulos, 28John, Nikolaidis,35John, Nikolaidis,35......John,Fissas,40John,Fissas,40 Query Result

Page 15: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Proxy StructureProxy Structure

PROXY STRUCTURE

MAIN CACHE

QUERY REWRITER

CACHE DIRECTORY

COOPERATIVE-SCHEME

DIRECTORY

WEIGHT CALCULATOR

REST OF COOPERATIVE SCHEME

URL/QTagTRANSFORMER

Page 16: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Proxy Structure – Proxy Structure –

URL/QTag TransformerURL/QTag Transformer

Proxies manipulates Web-Objects (Query Results) through Proxies manipulates Web-Objects (Query Results) through their <QTags>their <QTags>

Extract from a Web Object’s URL theExtract from a Web Object’s URL the

– Query (Knowing the CGI that produces the Query Query (Knowing the CGI that produces the Query Result)Result)

– LocationOfWebServerLocationOfWebServer

– DatabaseIDDatabaseID 1-1 correspondence between URLs and QTags1-1 correspondence between URLs and QTags

Page 17: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Proxy Structure – Proxy Structure –

Query RewriterQuery Rewriter

Rewriting the requested Web Objects (Queries) in case there is not Rewriting the requested Web Objects (Queries) in case there is not an exact match of the requested query cached but it can be an exact match of the requested query cached but it can be satisfied from other already cached web objects (queries).satisfied from other already cached web objects (queries).

Query rewriter will follow standard query-rewriting methods and Query rewriter will follow standard query-rewriting methods and techniques that are already used to database system and techniques that are already used to database system and environmentsenvironments

Page 18: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching Algorithm – Proposed Web Caching Algorithm – Proxy Structure – Proxy Structure – Weight CalculatorWeight Calculator

Every web object will be characterized from a Weight W which will be determined from the following Every web object will be characterized from a Weight W which will be determined from the following factors:factors:

  

SS (Determined from the web-object’s size)(Determined from the web-object’s size)

ΠΠss          (Determined from the influence percentage of Factor (Determined from the influence percentage of Factor S S to the Weight) to the Weight)

CSCS       (Determined from the web-object’s cost-retrieval)   (Determined from the web-object’s cost-retrieval)

ΠΠcscs          (Determined from the influence percentage of factor (Determined from the influence percentage of factor CSCS  to the Weight) to the Weight)

ΡΡ         (Determined from the web-object’s popularity)(Determined from the web-object’s popularity)

ΠΠpp        (Determined from the influence percentage of Factor (Determined from the influence percentage of Factor ΡΡ  to the Weight) to the Weight)

R R        (Determined from the web-object’s significance as far as its relationships (Determined from the web-object’s significance as far as its relationships concerns)concerns)

ΠΠrr          (Determined from the influence percentage of Factor (Determined from the influence percentage of Factor RR to the Weight) to the Weight)

Page 19: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Proposed Web Caching AlgorithmProposed Web Caching Algorithm

Some of the Sub-QTags are cached

None of the Sub-QTags are cached

The Request arrives to a Proxy

The URL/QTag Transformer Finds the QTag that best describes the incoming

URL

The QTag is sent to Query Rewriter

Query Rewriter Rewrites the Query and produces

Sub-QTags

The Query Rewriter asks the Cache Directory if any

of these Sub-QTags is already cached in the Main

Cache

All the Sub-QTags are

cached

Send request to Web Server and Caches

the response

Query Rewriter retrieves the locally cached Web Objects

The Query Rewriter asks the Cooperative-scheme Directory if the rest Sub-QTags cached in other

Proxies

Not all of the rest of the Sub-Qtags

are cached in other Proxies

ALL of the rest of the Sub-Qtags

are cached in other Proxies

Proxy retrieves the Cached Web Objects from the other Proxies

and sends them to Query Rewriter

Query Rewriter combines the Sub

QTags and the proxy sends the response

The Proxy Caches locally the retrieved

Web Objects

Weight Calculator Refreshes Weight Value and Parameters of

the Sub-Tags

Page 20: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Current and Future WorkCurrent and Future Work

Study and Testing the proposed new Study and Testing the proposed new approachesapproaches

Definition of WorkloadDefinition of Workload Better Definition and Testing of the Better Definition and Testing of the

proposed Algorithmproposed Algorithm

Page 21: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

DiscussionDiscussion

Efficiency of Testing Tools (Simulator)Efficiency of Testing Tools (Simulator) Ideas for efficient Web Caching for Ideas for efficient Web Caching for

Dynamic-DataDynamic-Data CommentsComments

Page 22: Dynamic-Content Web Caching with Cooperative Proxy Scheme Βελισκάκης Μανώλης Εθνικό Μετσόβιο Πολυτεχνείο Dept. of Electrical & Computer Engineering

Thank YouThank You