blank Quick help
blank Maintenance news

Scheduled maintenance

Regular maintenance outages:
between 05.00 and 05.15 hrs CET (Monday to Sunday).

Other outages
Availability

2022.02.11

More...
blank News flashes

News Flashes

New version of the European Patent Register – SPC proceedings information in the Unitary Patent Register.

2024-07-24

More...
blank Related links

Release notes
European Patent Register Assistant
European Patent Register Resource Book
European Patent Register – pocket guide
FAQ - European Patent Register
Federated Register (Flyer)
Global Dossier (Flyer)
Register events
Deep link coverage
Federated Register content
The Patent Information Tour
Fair use charter

Extract from the Register of European Patents

EP About this file: EP3079106

EP3079106 - SELECTING REINFORCEMENT LEARNING ACTIONS USING GOALS AND OBSERVATIONS [Right-click to bookmark this link]
StatusNo opposition filed within time limit
Status updated on  14.04.2023
Database last updated on 03.10.2024
FormerThe patent has been granted
Status updated on  06.05.2022
FormerGrant of patent is intended
Status updated on  10.11.2021
FormerExamination is in progress
Status updated on  20.05.2019
FormerRequest for examination was made
Status updated on  29.09.2017
Most recent event   Tooltip09.02.2024Lapse of the patent in a contracting state
New state(s): IT		published on 13.03.2024  [2024/11]
Applicant(s)For all designated states
DeepMind Technologies Limited
5 New Street Square
London EC4A 3TW / GB
[N/P]
Former [2020/17]For all designated states
DeepMind Technologies Limited
5 New Street Square
London
EC4A 3TW / GB
Former [2017/45]For all designated states
Google LLC
1600 Amphitheatre Parkway
Mountain View, CA 94043 / US
Former [2016/41]For all designated states
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043 / US
Inventor(s)01 / Schaul, Tom
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
02 / Horgan, Daniel George
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
03 / Gregor, Karol
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
04 / Silver, David
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
 [2021/47]
Former [2016/41]01 / SCHAUL, Tom
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
02 / HORGAN, Daniel George
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
03 / GREGOR, Karol
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
04 / SILVER, David
Belgrave House
76 Buckingham Palace Road
London, SW1W 9TQ / GB
Representative(s)Martin, Philip John
Marks & Clerk LLP
62-68 Hills Road
Cambridge
CB2 1LA / GB
[2022/23]
Former [2016/41]Derry, Paul Stefan, et al
Venner Shipley LLP
200 Aldersgate
London EC1A 4HD / GB
Application number, filing date16164072.706.04.2016
[2016/41]
Priority number, dateUS201562143677P06.04.2015         Original published format: US 201562143677 P
[2016/41]
Filing languageEN
Procedural languageEN
PublicationType: A2 Application without search report 
No.:EP3079106
Date:12.10.2016
Language:EN
[2016/41]
Type: A3 Search report 
No.:EP3079106
Date:29.03.2017
Language:EN
[2017/13]
Type: B1 Patent specification 
No.:EP3079106
Date:08.06.2022
Language:EN
[2022/23]
Search report(s)(Supplementary) European search report - dispatched on:EP23.02.2017
ClassificationIPC:G06N3/04, G06N3/08, G06N20/00
[2021/45]
CPC:
G06N3/08 (EP,US); G06N3/084 (CN); G06N20/00 (EP,CN,US);
G06N3/04 (CN); G06N3/045 (EP,CN,US)
Former IPC [2017/13]G06N3/04, G06N3/08, G06N99/00
Former IPC [2016/41]G06N3/04, G06N3/08
Designated contracting statesAL,   AT,   BE,   BG,   CH,   CY,   CZ,   DE,   DK,   EE,   ES,   FI,   FR,   GB,   GR,   HR,   HU,   IE,   IS,   IT,   LI,   LT,   LU,   LV,   MC,   MK,   MT,   NL,   NO,   PL,   PT,   RO,   RS,   SE,   SI,   SK,   SM,   TR [2017/44]
Former [2016/41]AL,  AT,  BE,  BG,  CH,  CY,  CZ,  DE,  DK,  EE,  ES,  FI,  FR,  GB,  GR,  HR,  HU,  IE,  IS,  IT,  LI,  LT,  LU,  LV,  MC,  MK,  MT,  NL,  NO,  PL,  PT,  RO,  RS,  SE,  SI,  SK,  SM,  TR 
TitleGerman:AUSWAHL VON AKTIONEN DES VERSTÄRKUNGSLERNENS UNTER VERWENDUNG VON ZIELEN UN BEOBACHTUNGEN[2021/46]
English:SELECTING REINFORCEMENT LEARNING ACTIONS USING GOALS AND OBSERVATIONS[2016/41]
French:SÉLECTION D'ACTIONS D'APPRENTISSAGE PAR RENFORCEMENT UTILISANT DES OBJECTIFS ET DES OBSERVATIONS[2021/46]
Former [2016/41]AUSWAHL VON BESTÄRKENDEN LERNMASSNAHMEN UNTER VERWENDUNG VON ZIELEN UND BEOBACHTUNGEN
Former [2016/41]ACTIONS D'APPRENTISSAGE PAR RENFORCEMENT DE SÉLECTION UTILISANT DES OBJECTIFS ET DES OBSERVATIONS
Examination procedure06.04.2016Date on which the examining division has become responsible
25.09.2017Amendment by applicant (claims and/or description)
25.09.2017Examination requested  [2017/44]
23.05.2019Despatch of a communication from the examining division (Time limit: M04)
05.09.2019Reply to a communication from the examining division
13.09.2021Cancellation of oral proceeding that was planned for 16.09.2021
16.09.2021Date of oral proceedings (cancelled)
11.11.2021Communication of intention to grant the patent
15.03.2022Fee for grant paid
15.03.2022Fee for publishing/printing paid
15.03.2022Receipt of the translation of the claim(s)
Opposition(s)10.03.2023No opposition filed within time limit [2023/20]
Fees paidRenewal fee
27.04.2018Renewal fee patent year 03
29.04.2019Renewal fee patent year 04
27.04.2020Renewal fee patent year 05
26.04.2021Renewal fee patent year 06
27.04.2022Renewal fee patent year 07
Opt-out from the exclusive  Tooltip
competence of the Unified
Patent Court		 See the Register of the Unified Patent Court for opt-out data
Responsibility for the accuracy, completeness or quality of the data displayed under the link provided lies entirely with the Unified Patent Court.
Lapses during opposition  TooltipAL08.06.2022
AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
IT08.06.2022
LT08.06.2022
LV08.06.2022
MC08.06.2022
PL08.06.2022
RO08.06.2022
RS08.06.2022
SI08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
IS08.10.2022
PT10.10.2022
[2024/11]
Former [2024/08]AL08.06.2022
AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
MC08.06.2022
PL08.06.2022
RO08.06.2022
RS08.06.2022
SI08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
IS08.10.2022
PT10.10.2022
Former [2023/25]AL08.06.2022
AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
PL08.06.2022
RO08.06.2022
RS08.06.2022
SI08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
IS08.10.2022
PT10.10.2022
Former [2023/17]AL08.06.2022
AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
PL08.06.2022
RO08.06.2022
RS08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
IS08.10.2022
PT10.10.2022
Former [2023/12]AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
PL08.06.2022
RO08.06.2022
RS08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
IS08.10.2022
PT10.10.2022
Former [2023/11]AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
PL08.06.2022
RO08.06.2022
RS08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
PT10.10.2022
Former [2023/09]AT08.06.2022
CZ08.06.2022
EE08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
RO08.06.2022
RS08.06.2022
SK08.06.2022
SM08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
PT10.10.2022
Former [2023/07]AT08.06.2022
ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
RS08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
Former [2022/50]ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
RS08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
Former [2022/49]ES08.06.2022
FI08.06.2022
HR08.06.2022
LT08.06.2022
LV08.06.2022
BG08.09.2022
NO08.09.2022
GR09.09.2022
Former [2022/47]ES08.06.2022
FI08.06.2022
LT08.06.2022
NO08.09.2022
Documents cited:Search[I]  - Frederick L Crabbe ET AL, "Goal Directed Adaptive Behavior in Second-Order Neural Networks: The MAXSON family of architectures", Adaptive Behavior, Thousand Oaks, CA, doi:10.1177/105971230000800204, (2000), pages 149 - 172, URL: http://journals.sagepub.com/doi/pdf/10.1177/105971230000800204, (20170214), XP055346179 [I] 1-15 * Sections 1 and 2 *

DOI:   http://dx.doi.org/10.1177/105971230000800204
 [A]  - VOLODYMYR MNIH ET AL, "Human-level control through deep reinforcement learning", NATURE, United Kingdom, (20150225), vol. 518, no. 7540, doi:10.1038/nature14236, ISSN 0028-0836, pages 529 - 533, XP055283401 [A] 1-15 * the whole document *

DOI:   http://dx.doi.org/10.1038/nature14236
 [A]  - THAM C K ED - FERRÁNDEZ JOSÉ MANUEL PAZ FÉLIX DE LA LOPE JAVIER DE, "Reinforcement learning of multiple tasks using a hierarchical CMAC architecture", ROBOTICS AND AUTONOMOUS SYSTEMS, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, (1995), vol. 15, no. 4, doi:10.1016/0921-8890(95)00005-Z, ISSN 0921-8890, pages 247 - 274, XP004001928 [A] 1-15 * Section 6 and 7 *

DOI:   http://dx.doi.org/10.1016/0921-8890(95)00005-Z
 [A]  - JUNFEI QIAO ET AL, "Q-learning Based on Neural Network in Learning Action Selection of Mobile Robot", AUTOMATION AND LOGISTICS, 2007 IEEE INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, (200708), ISBN 978-1-4244-1531-1, pages 263 - 267, XP031138775 [A] 1-15 * the whole document *
 [IP]  - TOM SCHAUL ET AL, "Universal Value Function Approximators", VOLUME 37: PROCEEDINGS OF THE 32ND INTERNATIONAL CONFERENCE ON MACHINE LEARNING, (20150711), XP055344798 [IP] 1-15 * the whole document *
by applicant   - SUTTON; RICHARD S; MODAYIL; JOSEPH; DELP; MICHAEL; DEGRIS; THOMAS; PILARSKI; PATRICK M, "Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction", THE TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS, (2011), vol. 2, pages 761 - 768
The EPO accepts no responsibility for the accuracy of data originating from other authorities; in particular, it does not guarantee that it is complete, up to date or fit for specific purposes.